trentmkelly/LessWrong-43k huggingface dataset:
Long Term Future Fund: April 2019 grant decisions
by habryka — 2019-04-08T02:05:44.217Z
Link: https://forum.effectivealtruism.org/posts/CJJDwgyqT4gXktq6g/long-term-future-fund-april-2019-grant-decisions

Please note that the following grants are only recommendations, as all grants are still pending an internal due diligence process by CEA.

This post contains our allocation and some explanatory reasoning for our Q1 2019 grant round. We opened up an application for grant requests earlier this year which was open for about one month, after which we received an unanticipated large donation of about $715k. This caused us to reopen the application for another two weeks. We then used a mixture of independent voting and consensus discussion to arrive at our current grant allocation.

What is listed below is only a set of grant recommendations to CEA, who will run these by a set of due-diligence tests to ensure that they are compatible with their charitable objectives and that making these grants will be logistically feasible.

Grant Recipients

Each grant recipient is followed by the size of the grant and their one-sentence description of their project.

Anthony Aguirre ($70,000): A major expansion of the Metaculus prediction platform and its community
Tessa Alexanian ($26,250): A biorisk summit for the Bay Area biotech industry, DIY biologists, and biosecurity researchers
Shahar Avin ($40,000): Scaling up scenario role-play for AI strategy research and training; improving the pipeline for new researchers
Lucius Caviola ($50,000): Conducting postdoctoral research at Harvard on the psychology of EA/long-termism
Connor Flexman ($20,000): Performing independent research in collaboration with John Salvatier
Ozzie Gooen ($70,000): Building infrastructure for the future of effective forecasting efforts
Johannes Heidecke ($25,000): Supporting aspiring researchers of AI alignment to boost themselves into productivity
David Girardo ($30,000): A research agenda rigorously connecting the internal and external views of value synthesis
Nikhil Kunapuli ($30,000): A study of safe exploration and robustness to distributional shift in biological complex systems
Jacob L. ($27,000): Building infrastructure to give X-risk researchers superforecasting ability with minimal overhead
Lauren Lee ($20,000): Working to prevent burnout and boost productivity within the EA and X-risk communities
Alex Lintz ($17,900): A two-day, career-focused workshop to inform and connect European EAs interested in AI governance
Orpheus Lummis ($10,000): Upskilling in contemporary AI techniques, deep RL, and AI safety, before purs

...
The Hard Work of Translation (Buddhism)
by romeostevensit — 2019-04-07T21:04:11.353Z

The issue, as it seems to me, is that almost every text you read on Buddhism does not attempt to do the actual work of translation. The first transmission of Buddhism to the west reified a bunch of translations of terms, such as concentration, equanimity, tranquility, mindfulness, suffering, etc. and works since then have mostly stuck to rearranging these words in different combinations and referencing the same metaphors that have been in use since the time of the Buddha. If these authors had true discernment they would realize that the umpteenth text on 'establishing the noble bases of tranquility secluded from sensuous ignorance' or what-have-you aren't helping anyone who didn't already get the message.

At this point I want to say that I think this approach is 'working' for the fraction of the population it is going to work for. If we want to make the practical fruits of Buddhist practice dramatically more accessible to a broader range of humanity we need people to do the hard work of translation to put the Buddha's teachings in forms that will be accessible to various groups of people.

The hard work of translation is to attempt to use language to point your mind at the same distinctions that the original author was trying to point to. Attempts to do this will inevitably fail in lots of ways, but can hopefully communicate enough of the core message that people can piece together the essential causal relations after which, having had direct experience as a result of skillful practice, they can help to improve the translations further.

So, putting my money where my mouth is, I want to try to produce a translation of what I see as the core causal loop that causes progress on the Buddha's path. I'm attempting this because I believe the core causal loop is actually quite small. The Buddha had a tougher task because he had to explain causation, locus of control, and other critical concepts to farmers from scratch.

To begin with, you may think that the purpose of meditation is to eliminate thoughts. But read the Pali Canon and you find a text rife with concepts, schemas, diagnostic methods for various classifications of mental activity, meditation taxonomies, sensory taxonomies, feedback loops etc. Pretending you're already enlightened and that there isn't hard work to do is something the new agers have borrowed...
Towards a Quieter Life
by Logan Riggs — 2019-04-07T18:28:15.225Z

My life is LOUD. Thoughts of fiction, conversations, unrealistic hypotheticals, games, tv shows flood my mind when there's no need to be babbling those thoughts. I have coroutines running in my mind to CHECK YOUR PHONE, YOUR EMAIL, YOUR RSS FEED, ...

Being aware of myself in those moments, I've realized I'm not actually happy doing any of them. Cravings for escapism is not what I want. I want something real, something true... I want reality.

So here's to a quieter life.

I'll check my phone/email after lunch and dinner. If I'm bored, I'll meditate. If I'm tired of meditating, I'll investigate that feeling. If that fails, I can Sabbath for a bit.

When I'm doing something, I'm only doing that. If an unrelated thought comes, I'll accept it and let it go. When I'm done with one thing, I'll go to the next with joy. If I don't have a next thing, I'll make a plan with joy. If that fails, I can Sabbath for a bit.

I'll be strict for the next few weeks, post the results, and go from there.

~Metta
Reinforcement learning with imperceptible rewards
by Vanessa Kosoy — 2019-04-07T10:27:34.127Z

TLDR: We define a variant of reinforcement learning in which the reward is not perceived directly, but can be estimated at any given moment by some (possibly costly) experiment. The reward function is no longer a function of the observation history, but a different object that we call "instrumental reward function". We give two definitions of instrumental reward function and prove their equivalence. We also derive a regret bound for this setting.

Background

In "classical" reinforcement learning the agent perceives the reward signal on every round of its interaction with the environment, whether through a distinct input channel or through some given way to compute the reward from the interaction history so far. On the other hand, we can rather easily imagine agents that optimize properties of their environment that they do not directly perceive. For example, if Alice, who lives in Dominica, donates money to the Against Malaria Foundation in order to save someone in Africa, then the result is usually not visible to Alice at the time it occurs, if ever. Similarly, Clippy the paperclip maximizer doesn't always perceive all the paperclips in the universe. Moreover, we might want to design agents that, in order to estimate the reward, direct queries to humans (which is costly and cannot be done continuously non-stop).

Now, it is possible to define the perceived reward as the subjective expected value of the "true" imperceptible reward (see the Results section for details). Although this transformation preserves expected utility, it does not preserve Bayesian regret. Indeed, Bayesian regret is the difference between the expected utility attained by the agent and the expected utility attained by a "reference" agent that knows the true environment from the onset. However, after the transformation, the reference agent will behave as if it knows the observable dynamics of the true environment but still pretends not to know the true environment for the purpose of computing the reward. Therefore, regret analysis requires us to consider the imperceptible reward honestly. In fact, as we will see, certain assumptions about the imperceptible reward function are needed even to make the problem learnable at all. Finally, this transformation makes the reward function more complex and hence applying a "generic" reinforcement learning algorithm after the transformation (instead of...
Factored Cognition with Reflection
by rmoehn — 2019-04-06T10:00:50.497Z

Note: I accidentally moved this post to draft and re-published it. My intention wasn't to get it back on the home page. Sorry.

Some weeks ago I completed an HCH-like program/Q&A system that supports reflection, including time travel: https://github.com/rmoehn/jursey I'm just posting it here in case someone finds it useful.
Alignment Newsletter #52
by Rohin Shah — 2019-04-06T01:20:02.232Z

Find all Alignment Newsletter resources

here

. In particular, you can

sign up

, or look through this

spreadsheet

of all summaries that have ever been in the newsletter.

Highlights

Thoughts on Human Models

(Ramana Kumar and Scott Garrabrant): Many approaches to AI safety involve modeling humans in some way, for example in order to correctly interpret their feedback. However, there are significant disadvantages to human modeling. First and most importantly, if we have AI systems do useful things without modeling humans, then we can use human approval as a "test set": we can check whether the AI's behavior is something we approve of, and this is an independent evaluation of the AI system. However, if the AI system had a human model, then it may have optimized its behavior for human approval, and so we cannot use approval as a "test set". Second, if our AI system has a catastrophic bug, it seems better if it doesn't have any human models. An AI system without human models will at worst optimize for some unrelated goal like paperclips, which at worst leads to it treating humans as obstacles and causing extinction. However, an AI system with human models with a catastrophic bug might optimize for human suffering, or having humans respond to email all day, etc. Thirdly, an AI system with human models might be simulating conscious beings that can suffer. Fourthly, since humans are agent-like, an AI system that models humans is likely to produce a subsystem that is agent-like and so dangerous.

The authors then discuss why it might be hard to avoid human models. Most notably, it is hard to see how to use a powerful AI system that avoids human models to produce a better future. In particular, human models could be particularly useful for interpreting specifications (in order to do what humans mean, as opposed to what we literally say) and for achieving performance given a specification (e.g. if we want to replicate aspects of human cognition). Another issue is that it is hard to avoid human modeling, since even "independent" tasks have some amount of information about human motivations in selecting that task.

Nevertheless, the authors would like to see more work on engineering-focused approaches to AI safety without human models, especially since this area is neglected, with very little such work currently. While MIRI does work on AI s...
Would solving logical counterfactuals solve anthropics?
by Chris_Leong — 2019-04-05T11:08:19.834Z

One of the key problems with anthropics is establishing the appropriate reference class. When we attempt to calculate a probability accounting for anthropics, do we consider all agents or all humans or all humans who understand decision theory?

If a tree falls on Sleeping Beauty argues probability is not ontologically basic and the "probability" depends on how you count bets. In this vein, one might attempt to solve anthropics by asking about whose decision to take a bet is linked to yours. You could then count up all the linked agents who observe A and all the agents who observe not A and then calculate the expected value of the bet. More generally, if you can solve bets, my intuition is that you can answer any other question that you would like about the decision by reframing it as a bet.
Rules and Skills
by Andrew Bindon — 2019-04-05T10:50:49.735Z

The extraordinary moment in 20th Century Philosophy when Ludwig Wittgenstein’s analysis of how people are able to follow rules lines up with Martin Heidegger’s exposition of skillfully dealing with the world we are embedded in.

… Thereby dismantling Epistemological Scepticism and Anti-realism in a single and simple insight.

https://thort.space/journey/110898789587140112950_5587971339654183211_6886495592041384105/1

There’s a lot going on in Ludwig Wittgenstein’s Philosophical Investigations and in Martin Heidegger’s Being and Time.

But a single and simple related move made by both Ludwig Wittgenstein and Martin Heidegger is enough to dismantle both … Epistemological Skepticism and Anti-realism…

https://thort.space/journey/110898789587140112950_5587971339654183211_6886495592041384105/2

These two master-works converge around the analysis of following a rule (L.W.) and skillfully dealing with the world we are embedded in (M.H.)

https://thort.space/journey/110898789587140112950_5587971339654183211_6886495592041384105/3

How do humans acquire skills?

By playing, experimenting, trial-and-error, trying things out
By practicing
By imitating
By being coached, instructed, trained
By being embedded in the world … we literally cannot help ourselves from acquiring skills, just as a consequence of the continuous feedback that the world gives us all the time, moment by moment, as we live our lives !

https://thort.space/journey/110898789587140112950_5587971339654183211_6886495592041384105/4

Elaboration of Coaching

The coach breaks skills down into sub-skills and trains the apprentice in each of the sub-skills , and then trains the apprentice to combine the sub-skills

(1) The coach says: Do-X … the apprentice tries to do X.
(2) The coach tells the apprentice whether they did X, or how well or badly they did X
(3) repeat

Notice that the apprentice does not know whether they are doing X or not, or doing it correctly, or doing it well or doing it badly — apart from the say-so of the coach.

https://thort.space/journey/110898789587140112950_5587971339654183211_6886495592041384105/5

Elaboration of being embedded in the world

We are coupled with the world … the world gives us continuous feedback in respect of the consequences and outcomes of our actions.

At one time you couldn’t do it … now you have to really concentrate or pay attention to not do it !

https://thort.space/journey/110898789587140112950_5587...
How would a better replacement for tissues look like?
by ChristianKl — 2019-04-05T09:13:36.335Z

When one has a cold common disposable tissues are the state of the art solution for getting mucus out of one's nose.

If there's however a lot of mucus a bit of that mucus usually ends up on the hand that holds the tissue.

This in turn makes it likely that germs end up on the hand and on objects that one touches with the hand.

If we invent a better solution to replace disposable tissues, it would be valuable for reducing the spread of infectious disease.

Does anybody have ideas?
Aumann Agreement by Combat
by roryokane — 2019-04-05T05:15:44.431Z
Link: http://sigbovik.org/2019/proceedings.pdf#page=8

The first paper in the SIGBOVIK 2019 proceedings, “Aumann Agreement by Combat” by Travis Hance, seems relevant to this site. The paper is on page 4 of the PDF (page 8 if you include front matter).

The abstract:

The celebrated Aumann’s Agreement Theorem shows that two rational agents with the same priors on an event who make different observations will always converge on the same posteriors after some civilized conversation over tea. Furthermore, they will come to agree even if they do nothing other than simply state their posteriors over and over again.

However, this protocol is widely criticized for being too boring. We therefore introduce a more exciting alternative, which we name Aumann Agreement by Combat.

The SIGBOVIK conference is held every year on April 1.
How do people become ambitious?
by Raemon — 2019-04-04T19:12:26.826Z

I have a pet theory about how people become ambitious and agenty.

I was about to write up some insight porn about it, and then was like "you know, Raemon, you should probably actually think about about this for real, since it seems like Pet Psychology Theories are one of the easier ways to get stuck in dumb cognitive traps."

"How do people get ambitious and agenty" seems to be something people should have actually tried to study before. I'm thinking something as simple as "interviewing lots of people and checking for common patterns."

2 seconds spent on google scholar suggested I could use better keywords.

Curious if anyone has looked into this (either reviewing existing literature, or conducting interviews themselves or otherwise trying to tackle the question in a serious way)

For clarity, the two phenomena I want to understand better are:

Ambition. How do people end up having plans (which they realistically expect to achieve) that affect at least thousands, and preferably millions of people?
Agency. How do people generally gain the capacity to be self-motivated, think through plans, and decide how to pursue goals that will change the world (changing it in small ways is fine)
Modelling Model Comparisons
by Logan Riggs — 2019-04-04T17:26:45.565Z

Define a model as a set of objects and their relationships. For example, when discussing a model of music, "notes" would be the objects and a possible relationship would be "harmony". [Technically, the objects would be (frequency, volume, time), and from their you could define the relationships "harmony", "tempo", "crescendo", etc.]

Using this, there are two different ways to compare models: similar relationships and (Object1, Relationship) = (Object2)

1. Similar Relationships [Type 1 comparison]

Using the relationship "Repetition", there is repetition in musical phrases (Fur Elise), in poetic phrases (Annabel Lee), in song chorus's, themes in movies, etc. Even though these four examples contain different objects, we're able to find the similar relationships and compare them.

I can think of two uses for this type of comparison. The first is in metaphors. The second is in generalization. Elaborating on the latter, I find a piece of music annoying if it's too repetitive. The same is true, for poems, songs, and movies; however, I very much enjoy a song if it strikes a good balance between repetition and novelty. Learning to strike that balance when writing music has generalized to doing the same in writing lyrics and dance choreography.

The same generalization can happen in non-art fields as well, such as realizing that two different functions are both convex or that two different problems can be solved recursively, and so on.

2. (Object1, Relationship) = (Object2) [Type 2 comparison]

A good example to start is:

(Object1,Relationship)=(Object2)(Carbon, atoms arranged in diamond cubic)=("diamond")

But this is more than just connecting quarks/electrons to atoms or atoms to objects in language. You can connect models of various levels together. With the model of language, we can connect to the low-level object "quarks" and the high-level object "diamond". This has helped my understanding of how Multi-level Models relate to transparency: if an AI can find the correct Type 2 comparison between what it's doing and our human language, then transparency is solved; however, human language is very complex.

Human Language and Communicating Models

Let's say you're talking with Alice. Two failures in communicating models are when you're (1) not discussing the same object...
IRL 5/8: Maximum Causal Entropy IRL
by RAISE — 2019-04-04T10:53:31.792Z
Link: https://app.grasple.com/#/level/1542

Every Monday for 8 weeks, we will be posting lessons about Inverse Reinforcement Learning. This is lesson 5. We are publishing it now because it would have interfered with the randomized controlled trial we are running, and yesterday we finished collecting responses from the participants. A LW post with the results will appear in a few days. Future IRL lessons will resume normally on Monday.

Note that access to the lessons requires creating an account here.

This lesson comes with the following supplementary material:

The Principle of Maximum Causal Entropy

Have a nice day!
Ideas ahead of their time
by Theist — 2019-04-03T22:15:20.673Z

Imagine a person in the ancient world who came up with the following idea: "What would the sun and moon look like if they were very very far away?" This idea would likely lead to the conclusion that they would look like tiny points of light, which then could lead to the question "What if the tiny points of light we call stars and planets are actually faraway suns and moons?"

Unfortunately, our ancient friend would likely be stuck at that point, due to the limitations of human vision and the lack of proper instruments for examining the nature of celestial objects. But our friend would be right, unlike nearly every other human until Giordano Bruno's cosmology of 1584.

My questions then are, what other ideas of similar power exist, how will we know them if we find them, and is there any way to search for them intentionally?
On AI and Compute
by johncrox — 2019-04-03T19:00:09.899Z

This is a post on OpenAI’s “AI and Compute” piece, as well as excellent responses by Ryan Carey and Ben Garfinkel, Research Fellows at the Future of Humanity Institute. (Crossposted on the EA Forum)

Intro: AI and Compute

Last May, OpenAI released an analysis on AI progress that blew me away. The key takeaway is this: the computing power used in the biggest AI research projects has been doubling every 3.5 months since 2012. That means that more recent projects like AlphaZero have tens of thousands of times the “compute” behind them as something like AlexNet did in 2012.

When I first saw this, it seemed like evidence that powerful AI is closer than we think. Moore’s Law doubled generally-available compute about every 18 months to 2 years, and has resulted in the most impressive achievements of the last half century. Personal computers, mobile phones, the Internet...in all likelihood, none of these would exist without the remorseless progress of constantly shrinking, ever cheaper computer chips, powered by the mysterious straight line of Moore’s Law.

So with a doubling cycle for AI compute that’s more than five times faster (let’s call it AI Moore’s Law), we should expect to see huge advances in AI in the relative blink of an eye...or so I thought. But OpenAI’s analysis has led some people to the exact opposite view.[1]

Interpreting the Evidence

Ryan Carey points out that while the compute used in these projects is doubling every 3.5 months, the compute you can buy per dollar is growing around 4-12 times slower. The trend is being driven by firms investing more money, not (for the most part) inventing better technology, at least on the hardware side. This means that the growing cost of projects will keep even Google and Amazon-sized companies from sustaining AI Moore’s Law for more than roughly 2.5 years. And that’s likely an upper bound, not a lower one; companies may try keep their research budgets relatively constant. This means that increased funding for AI research would have to displace other R&D, which firms will be reluctant to do.[2] But for lack of good data, for the rest of the post I’ll assume we’ve more or less been following the trend since the publication of “AI and Compute”.[3]

While Carey thinks that we’ll pass some interesting milestones for compute during this time which might be promising for research, Ben Garfinkel is much more pessimistic. Hi...