trentmkelly/LessWrong-43k huggingface dataset:
Imagine a World Where Governments Treated COVID-19 Properly
by jdcampolargo — 2020-08-12T22:52:55.125Z

This essay was originally published on my website.

Over the last few weeks, I’ve reaffirmed my position that a statistics class should be REQUIRED for every person before they are released into the open world of adulthood.

I was first introduced to the marvelous field of statistics when I was selected to attend a program at Argonne National Laboratory, a science and engineering research laboratory where interesting developments take place ranging from supercomputers to nanotechnology.

In this program, my team and I needed to figure out where the 1850s cholera outbreak broke out. We were given a set of tools such as statistics, coding, databases, machine learning, and many others. At the end of the program, my team found the exact street and source of the spreading of cholera. I was hooked.

I’m a voracious reader, and Forbes was one magazine I used to read monthly. One time, there was a very successful person who wrote about his life and work. He said one thing that I always remember. He mentioned that if he could do high school all over again, he would take a statistics course such as AP Statistics, and lessen his focus over classes such as calculus. He said statistics was critical to his success and the workforce would need more people who understand statistics because it is very hard to automate and human interpretation would always need to be there.

I took his advice and took AP Statistics during my senior year of high school. This class quickly became my favorite, and my appreciation and fascination only kept increasing and increasing. I can pursue and create so many businesses and projects with the help of statistics. Statistics makes me see clearly.

After seeing everything that has happened, it is dangerous to have a large number of people who don't know or understand statistics and the concept of probabilities. Many people are fooled by noise, lack a sense of judgment, and blindly believe what they see or what others say.

Statistics is KEY.

Why is it difficult for people to understand statistics and probabilities? Let’s try to understand why that happens.

According to Richard Peterson’s Inside the Investor’s Brain, “When an outcome is possible but not probable, people tend to overestimate its chances of occurring. This is called the possibility effect…Emotions in u...
Investment idea: basket of tech stocks weighted towards AI
by ioannes — 2020-08-12T21:30:58.261Z

I'm not an investment advisor and this isn't investment advice. Excerpted from an occasional newsletter I write about investing.

Related: Engaging Seriously with Short Timelines, You Need More Money

Recent developments in AI (AlphaGo Zero, AlphaFold, GPT-3) make transformative AI seem plausible within the next 10-50 years.

There's no fire alarm for AGI, so now is as good a time as any to have my portfolio reflect this belief. (A little embarrassing that I've waited so long to follow my conviction here! The joys of compartmentalized cognition...)

If transformative AI follows the trend currently being set by OpenAI, where it's expensive to initially train an AI model and cheap to deploy it once trained, profits will most likely be captured by the companies that train the AIs & the value chain supporting these companies (OpenAI's API is an early version of this; see also Gwern's fantastic analysis of the strategies of various AI research groups).

If training costs aren't the limiting factor, it's less clear that profits will be captured by the companies that own the models. This scenario feels higher variance & weirder... probably the big tech companies still benefit a lot (they're already set up to deliver services at scale and they can just buy smaller firms who might otherwise compete), and makers of GPUs & semiconductors probably still benefit as well. Peter McCluskey has a good rundown of sectors that could benefit.

Happily these two scenarios are pretty aligned from a small-investor perspective – apart from OpenAI, a lot of AI development & the supporting value chain is housed within big tech companies, and even OpenAI runs on Microsoft compute.

Finally (and more weirdly), I think now is a good time to invest in "shamanic" leaders (h/t Max for the term). It seems like a lot of society doesn't really know how to orient anymore, so when someone comes along with a clear + compelling vision, they can raise a lot of capital. Bezos & Musk use this dynamic to power their companies; Neumann & Holmes exploited it.

This shamanic consideration isn't driving my decision-making but I take it as a plus when I see it. Amazon > Microsoft. Facebook > Google. It's a bad sign when I can't remember the name of the current CEO.

Here's how the allocation looks:

40% F

...
Blog post: A tale of two research communities
by Aryeh Englander — 2020-08-12T20:41:30.448Z

This is a copy of a blog post from Francis Rhys Ward, an incoming doctoral student in Safe and Trusted AI at Imperial College London. I just discovered this on the website of the Assuring Autonomy International Programme and thought it was worth cross-posting here.

The post is also available on Medium.

(Francis, if you're reading this, thanks for the shout-out!)

Blog post: A tale of two research communities

POSTED ON TUESDAY 11 AUGUST 2020

What “AI safety” means to them both and steps to collaboration

The term “AI safety” means different things to different people. Alongside the general community of artificial intelligence (AI) and machine learning (ML) researchers and engineers, there are two different research communities working on AI safety:

The assuring autonomy/safety engineering field: a community of experts with a long history in assuring real-world autonomous systems (not just AI and ML!)
The AI safety/longtermist community: a relatively new field focused on the consequences of transformative AI (TAI), artificial general intelligence (AGI), and smarter than human AI

Having worked at the

Assuring Autonomy International Programme (AAIP)

, and now researching how AI can learn human values at Imperial College London, I am interested in how both communities can learn and benefit from each other. Here I’ll give a brief overview of these fields, point to some key resources within them, and try to highlight the benefits of integrating these communities. This post is primarily intended for researchers from either field who wish to gain insight into the other.

Assuring autonomy

The assuring autonomy community is part of the wider field of safety engineering for complex systems. This community focuses on practical issues surrounding current and near-term technologies, with the typical approach being to set safety requirements for a system and generate evidence that requirements have been met, including integrated utilisation of expert knowledge, testing, verifying, validating, as well as regulation.

Historically, work has been done on problems related to autonomous systems in general (such as assistive flight technologies) and more recently the field has shifted focus to AI and ML (e.g. with the

AAIP

,

Darpa’s Assured Autonomy Program

, and more recent

Johns Hopkins Institute for Assured Autonomy

).

Work such as

Assuring the

...
Rationally Ending Discussions
by curi — 2020-08-12T20:34:14.951Z

Discussions should end gracefully. There should be some clarity about why they’re ending and why that’s rationally acceptable. If someone wants to continue the discussion, they should have the opportunity to see why this ending is reasonable. (Or, failing that, they should have some evidence available to enable them to argue their case that it was unreasonable. They shouldn’t be left with little or no data about what happened.)

If you discuss poorly, it’s important that you can learn from that and do better next time. If you want it, you should have access to some critical feedback, some explanation of the other person’s perspective, or something to enable progress. If you’re running into problems interacting with people, but no one will tell you what the problem is, that’s bad for rational progress.

The more easily discussions can end, the more easily they can start. We don’t want discussing to be a big burden or commitment (it’s OK if a few discussions require high effort, when people have reason to voluntarily choose that, but that shouldn’t be the default).

Discussions can end by mutual agreement. If no one objects to the ending, that’s fine. That’s one way to end discussions. Pretty much everyone agrees on this. The difficulty is that sometimes people don’t agree. Someone wants to stop discussing but someone else wants to pursue the matter it further.

I don’t think agreeing to disagree, by itself, is a good reason to end a discussion by mutual agreement. Disagreements can be resolved! There should be an extra factor like agreeing that it’s reasonable to prioritize other stuff (which we may leave implied without explicitly mentioning). There are many problems to work on, and time/energy/etc are scare resources, so it’s fine to drop a disagreement (for now, indefinitely) if people think they can get more value working on other stuff.

Ending discussions when a single person wants to, for any reason, with no explanation, is problematic. For example, people will end discussions when they (the ideas they are biased in favor of) start losing the argument.

But we don’t want to trap people in discussions. Sometimes one guy has a lot of energy to discuss something forever but you want to stop. There are lots of legitimate reasons to end a discussion.

You can try to explain your reasons for ending a discussion, but no matter what you say, the other guy might disagree. Th...
Money creation and debt
by AnthonyC — 2020-08-12T20:30:42.321Z

I'm confused about fiat money creation and debt in the US. My mental model is that the Fed lends money into existence (either to banks or to the treasury), which creates a loan (that eventually has to be paid back with interest, though I think they can choose not to collect on loans to the treasury?) (I'm also not sure where the money the Fed uses to pay interest on bank reserves comes from, but since it used to not do that I'll ignore it and pretend I'm asking about the time before that existed). The fractional reserve banking system then creates a convergent geometric series of additional money creation through repeated lending and saving, and that's basically M2. (I'm ignoring the fact that the reserve requirement is currently zero, I don't think it affects my question anyway).

It seems like that means that for every dollar that exists, there is an equivalent dollar owed in debt to someone. And conversely, when debt gets paid down, the monetary base decreases. This suggests that it is not possible, economy-wide, for private sector net savings to increase unless the government debt increases, and that whenever government debt goes down, private savings must decrease. I feel like I'm missing something central, because otherwise this would be public policy 101 level stuff that the Left especially would be very vocal about, but I can't figure out what.
How GPT-N will escape from its AI-box
by hippke — 2020-08-12T19:34:48.410Z

Scenario: A human user asks GPT-N a plain text question through the usual browser textbox interface. GPT-N answers in the form of text, which contains clever Javascript code. Instead of being displayed, the malicious code triggers a browser exploit and gets executed. Now GPT-N has an executable of its choice running on the client side and can proceed from there. It has left its AI box with only a text-interface, without asking the user to let it out, and possibly without the user even noticing. Perhaps GPT-3 has done this already, while "deliberately" pretending to be stupider than it is?
Strong implication of preference uncertainty
by Stuart_Armstrong — 2020-08-12T19:02:50.115Z

Here is a theory that is just as good as general relativity:

AGR (Angel General Relativity): Tiny invisible angels push around all the particles in the universe in a way that is indistinguishable from the equations of general relativity.

This theory is falsifiable, just as general relativity (GR) itself is. Indeed, since it gives exactly the same predictions as GR, a Bayesian will never find evidence that prefers it over Einstein's theory.

Therefore, I obviously deserve a Nobel prize for suggesting it.

Enter Occam's shaving equipment

Obviously the angel theory is not a revolutionary new theory. Partially because I've not done any of the hard work, just constructed a pointer to Einstein's theory. But, philosophically, the main justification is Occam's razor - the simplest theory is to be preferred.

From a Bayesian perspective, you could see violations of Occam's razor as cheating, using your posterior as priors. There is a whole class A of "angels are pushing particles" theories, and AGR is just a small portion of that space. By considering AGR and GR on equal footing, we're privileging AGR above what it deserves[1].

In physics, Occam's razor doesn't matter for strictly identical theories

Occam's razor has two roles: the first is to distinguish between strictly identical theories; the second is to distinguish between theories that give the same prediction on the data so far, but may differ in the future.

Here, we focus on the first case: GR and AGR are strictly identical; no data will ever distinguish them. In essence, the theory that one is right and the other wrong is not falsifiable.

What that means is that, though AGR may be a priori less likely than GR, the relative probability between the two theories will never change: they make the same predictions. And also because they make the same predictions, that relative probability is irrelevant in practice: we could use AGR just as well as GR for predictions.

How preferences differ

Now let's turn to preferences, as described in our paper "Occam's razor is insufficient to infer the preferences of irrational agents".

Here two sets of preferences are "prediction-identical", in the sense of the physics theories above, if they predict the same behaviour for the agent. So that means that two different preference-based explanations for the same behaviour will never change their relative probabilities.

Worse than that, O...
Alignment By Default
by johnswentworth — 2020-08-12T18:54:00.751Z

Suppose AI continues on its current trajectory: deep learning continues to get better as we throw more data and compute at it, researchers keep trying random architectures and using whatever seems to work well in practice. Do we end up with aligned AI “by default”?

I think there’s at least a plausible trajectory in which the answer is “yes”. Not very likely - I’d put it at ~10% chance - but plausible. In fact, there’s at least an argument to be made that alignment-by-default is more likely to work than many fancy alignment proposals, including

IRL variants

and

HCH-family methods

.

This post presents the rough models and arguments.

I’ll break it down into two main pieces:

Will a sufficiently powerful unsupervised learner “learn human values”? What does that even mean?
Will a supervised/reinforcement learner end up aligned to human values, given a bunch of data/feedback on what humans want?

Ultimately, we’ll consider a semi-supervised/transfer-learning style approach, where we first do some unsupervised learning and hopefully “learn human values” before starting the supervised/reinforcement part.

As background, I will assume you’ve read some of the core material about human values from the sequences, including

Hidden Complexity of Wishes

,

Value is Fragile

, and

Thou Art Godshatter

.

Unsupervised: Pointing to Values

In this section, we’ll talk about why an unsupervised learner might not “learn human values”. Since an unsupervised learner is generally just optimized for predictive power, we’ll start by asking whether theoretical algorithms with best-possible predictive power (i.e. Bayesian updates on low-level physics models) “learn human values”, and what that even means. Then, we’ll circle back to more realistic algorithms.

Consider a low-level physical model of some humans - e.g. a model which simulates every molecule comprising the humans. Does this model “know human values”? In one sense, yes: the low-level model has everything there is to know about human values embedded within it, in exactly the same way that human values are embedded in physical humans. It has “learned human values”, in a sense sufficient to predict any real-world observations involving human values.

But it seems like there’s a sense in which such a model does not “know” human values. Specifically, although human values are embedded in the low-level model, the embedding itself is nontrivial. Eve...
Matt Botvinick on the spontaneous emergence of learning algorithms
by Adam Scholl — 2020-08-12T07:47:13.726Z

Matt Botvinick is Director of Neuroscience Research at DeepMind. In this interview, he discusses results from a 2018 paper which describe conditions under which reinforcement learning algorithms will spontaneously give rise to separate full-fledged reinforcement learning algorithms that differ from the original. Here are some notes I gathered from the interview and paper:

Initial Observation

At some point, a group of DeepMind researchers in Botvinick’s group noticed that when they trained a RNN using RL on a series of related tasks, the RNN itself instantiated a separate reinforcement learning algorithm. These researchers weren’t trying to design a meta-learning algorithm—apparently, to their surprise, this just spontaneously happened. As Botvinick describes it, they started “with just one learning algorithm, and then another learning algorithm kind of... emerges, out of, like out of thin air”:

"What happens... it seemed almost magical to us, when we first started realizing what was going on—the slow learning algorithm, which was just kind of adjusting the synaptic weights, those slow synaptic changes give rise to a network dynamics, and the dynamics themselves turn into a learning algorithm.”

Other versions of this basic architecture—e.g., using slot-based memory instead of RNNs—seemed to produce the same basic phenomenon, which they termed "meta-RL." So they concluded that all that’s needed for a system to give rise to meta-RL are three very general properties: the system must 1) have memory, 2) whose weights are trained by a RL algorithm, 3) on a sequence of similar input data.

From Botvinick’s description, it sounds to me like he thinks [learning algorithms that find/instantiate other learning algorithms] is a strong attractor in the space of possible learning algorithms:

“...it's something that just happens. In a sense, you can't avoid this happening. If you have a system that has memory, and the function of that memory is shaped by reinforcement learning, and this system is trained on a series of interrelated tasks, this is going to happen. You can't stop it."

Search for Biological Analogue

This system reminded some of the neuroscientists in Botvinick’s group of features observed in brains. For example, like RNNs, the human prefrontal cortex (PFC) is highly recurrent, and the RL and RNN memory systems in their meta-RL model reminded them of “synaptic m...
You Need More Money
by gilch — 2020-08-12T06:16:09.827Z

Part 1 of the Inefficient Markets Sequence.

[Epistemic Disclaimer:
I am not rich yet.
I feel like I'm just barely starting to understand this stuff
and perhaps that is the best time to teach it:
while I still remember what wasn't obvious,
both to cultivate my comprehension,
and to enrich the rationalists.

For educational purposes only!
Double-check anything I say.
Read the comments.
You have been warned!

You are responsible for your own money.
Do your due diligence and don't rely too much on random internet bloggers :)
I do not know your financial situation.
I am not your financial advisor.]

Convergent Instrumental Rationality

Sufficiently advanced intelligent agents,
almost no matter their ultimate ends,
will tend to pursue instrumental means such as self-preservation and resource acquisition.

What of sufficiently advanced humans?

The Outside Perspective

Can you imagine a world you'd rather live in?
Perhaps your life is comfortable, for the moment.
But the world is truly awful right now.
Children are dying as we speak, of hunger, or war, or disease.
A cheap shot, but I'm sure you could come up with many other ills.
The world was even worse, in many ways, in the recent past.
It could become worse again, or better.
Perhaps a lot better.

If you truly have something to protect, what are your means?

If, when pondering a question,
you discover what something smarter than yourself would answer,
perhaps you have found your answer also.

What would a superior intelligence do in your shoes?
What would a more advanced culture think of ours?

You can't know all the answers to these questions,
but we do know this much: acquire resources.

"Need" Is a Relative Term.

Distinguishing a "need" from a "want"
is one of those elementary-school tasks that we all think is easy, at the time.
It's a basic budgeting skill:
buy what you need first,
then buy what you want with what's left over.

But upon deeper examination, it's not so simple.

Do you need to see that doctor?
Your ancestors only a few generations back lacked access to competent medicine.
The physicians of the past often did more harm than good.
Were their basic needs being met?
And in the more recent past, many diseases which now have effective cures were fatal.
Were their basic needs being met?
Not by today's standards.

From the perspective of a more advanced culture (if we knew more, thought faster, were more the people we wished w...
Zen and Rationality: Trust in Mind
by Gordon Seidoh Worley — 2020-08-11T20:23:34.434Z

This is post 2/? about the intersection of my decades of LW-style rationality practice and my several years of Zen practice.

In today's installment, I look at "trust in mind" from a rationalist perspective.

I really like Trust in Mind, a Zen teaching poem attributed to the Third Patriarch, because it offers a vivid description of Zen practice that I've kept coming back to for new guidance and insight. Today I only want to talk about its origin and its title, because to delve in much more would be to write a book on Zen.

Purportedly it was written in response to the rising popularity of Pure Land Buddhism in an attempt to encourage the people to practice the "true" Buddha way, and revolves around the idea of putting trust in mind rather than trust, as in the case of Pure Land Buddhism, in a deified Buddha that will lead people to the Pure Land if you chant his name. This seems a deeply humanist message, and from a certain perspective this is fair: Zen contains a strong humanist aspect in that it's all about doing the practice here and now, and who else is there to do the practice but you, a human? But this is where the cracks in the humanist interpretation start to show, because Zen will immediately turn around and ask "oh, so where's this 'you' that's doing the practice?" and "does a dog have Buddha-nature?". Zen's tricky like that, offering you something with one hand while pulling the rug out from under you with the other, repeatedly trying to get you to look up and see what you've been leaving out.

Turning to the title, the "mind" here is again the heart-mind or heart-body-mind. This is an idea that comes up a lot in Zen, so it's worth saying a bit about it. Partly it reflects the traditional idea that a person's emotions or soul reside in their heart rather than their head, and the use of shin/心 to mean heart and mind is common in classical Chinese writing, not just in Zen/Ch'an texts. But if we're charitable and ask why would someone without the benefit of modern medicine think an import part of themselves resided in the heart, I reckon it's deeply related to the way we talk about feeling our emotions. That is, most people report experiencing emotions and sometimes general thought somatically, i.e. in the body. Sometimes this ability to literally feel emotions is impaired, and even when it's not we can get better at it through focusing on the mind-body-"hea...