trentmkelly/LessWrong-43k huggingface dataset:
Have You Tried Hiring People?
by rank-biserial — 2022-03-02T02:06:39.656Z

The purpose of this post is to call attention to a comment thread that I think needs it.

seed:

Wait, I thought EA already had 46$ billion they didn't know where to spend, so I should prioritize direct work over earning to give? https://80000hours.org/2021/07/effective-altruism-growing/

rank-biserial:

I thought so too. This comment thread on ACX shattered that assumption of mine. EA institutions should hire people to do "direct work". If there aren't enough qualified people applying for these positions, and EA has 46 billion dollars, then their institutions should (get this) increase the salaries they offer until there are.

To quote EY:

There's this thing called "Ricardo's Law of Comparative Advantage". There's this idea called "professional specialization". There's this notion of "economies of scale". There's this concept of "gains from trade". The whole reason why we have money is to realize the tremendous gains possible from each of us doing what we do best.

This is what grownups do. This is what you do when you want something to actually get done. You use money to employ full-time specialists.

Excerpts From The ACX Thread:[1]

Daniel Kokotajlo:

For those trying to avert catastrophe, money isn't scarce, but researcher time/attention/priorities is. Even in my own special niche there are way too many projects to do and not enough time. I have to choose what to work on and credences about timelines make a difference.

Daniel Kirmani:

I don't get the "MIRI isn't bottlenecked by money" perspective. Isn't there a well-established way to turn money into smart-person-hours by paying smart people very high salaries to do stuff?

Daniel Kokotajlo:

My limited understanding is: It works in some domains but not others. If you have an easy-to-measure metric, you can pay people to make the metric go up, and this takes very little of your time. However, if what you care about is hard to measure / takes lots of time for you to measure (you have to read their report and fact-check it, for example, and listen to their arguments for why it matters) then it takes up a substantial amount of your time, and that's if they are just contractors who you don't owe anything more than the minimum to.

I think another part of it is that people just aren't that motivated by money, amazingly. Consider: If the prospect of getting paid a six-figure salary to solve technical alignment problems worked to...
How to develop safe superintelligence
by martillopart — 2022-03-01T21:57:22.811Z

Author: Martí Llopart

Introduction

In the world we live in, there are many problems that need to be urgently solved: climate change[1], the risk of a new pandemic[2], wealth inequality[3], the aging population[4] and more. All of these problems are hugely complex and solving them requires vast amounts of intelligence. In order to address that, there’s a tool that comes in very handy: Artificial Intelligence. We are all familiar with AI, but to put it shortly, it’s the intelligence demonstrated by machines[5]. In recent years, AI has proven to be a game changer for industries such as healthcare and finance, but most experts agree that we are just at the beginning of what has been termed as “The AI revolution”[6]. The culmination point of this AI revolution is the creation of a Superintelligence, a hypothetical AI agent that possesses intelligence far surpassing that of the brightest and most gifted human minds. If this Superintelligence is aligned with our goals, a solution could be found to the most complex scientific, social and economical problems, leading to huge benefits for humanity. But before creating Superintelligence, General Intelligence must be reached. Artificial General Intelligence (AGI) is the hypothetical intelligence of a machine that has the capacity to understand or learn any intellectual task that a human being can[7]. It is just one step below Superintelligence. So, how do we get to AGI? In this essay, divided in three parts, I’ll try to answer this question.

Part 1: Creating the best general ML algorithm

Our brains are general problem solvers. Their objective is to use the information from the senses to accurately predict future events and to act according to these predictions in order to survive [8]. If we want to develop General Intelligence, we must first create the best general learning algorithm (GLA), an algorithm capable of performing well at learning any type of task. The best GLA would be the one that performs perfectly at learning any task.

However, there’s a big problem in developing such an algorithm: research in machine learning lacks a theoretical backbone. In the last decade, most of the advances in AI have been powered by neural networks, but the theory behind them still remains poorly understood. Although the math behind each individual neuron is well understood, the mathematical theory of the emergen...
What are the causality effects of an agents presence in a reinforcement learning environment
by Jonas Kgomo — 2022-03-01T21:57:15.928Z

Suppose we have an agent A trying to optimise for a reward R in an environment S.
How can we tell that the presence of the agent does not affect the environment and the measurement(observation) is not only subject to the agent but the environment?

This is related with the measurement problem in quantum computing, we have an agent (a particle ) entangled in a quantum superposition, consider an electron with two possible configurations: up and down, a|↑⟩+b|↓⟩, when we measure the state, it collapses the wavefunction to a particular classical state, up or down.
Moreover, the observer effect, notes that measurements of certain systems cannot be made without affecting the system.
While the uncertainty principle argues that we cannot predict the value of a quantity with arbitrary certainty.

Another way to state the problem is, does measuring the state of the action affect the state of the environment?
How are the observations in physics different from the observations we make in RL?
Is the environment state at S1 causal to the state in S2 ?
What are some ways to do a PhD without an educational institution
by Jonas Kgomo — 2022-03-01T21:57:15.896Z

Is it possible to get a PhD from one of the research institutions, for example the PhD residency is pretty good initiative but hard to get into.

What are some examples of getting a PhD the non-traditional way.

Nadia Eghbal posted a blog about this https://nadia.xyz/phd
What are some low-cognitive -workload tasks that can help improve the world?
by Yitz — 2022-03-01T17:47:05.140Z

I have a chronic health condition which results in significant brain fog, usually lasting a few hours most days. When I don’t have brain fog I can work on high-level technical work with relative ease, but when I’m experiencing symptoms, even simple tasks become a challenge (and manual labor is out of the question). As a result, most activities I’d like to do in the Effective Altruism space are out of reach for a good portion of my day. What are some ways to put that time to optimal use?

Beyond myself, it would be nice to have a list of low-cognitive-workload altruistic tasks to refer people to if they don’t have the intellectual capability to directly work on the alignment problem or whatever, but still want to help make the world a better place.
The Age of Over-population (Chapter 4)
by Justin Bullock — 2022-03-01T14:10:39.697Z

"In the Brave New World of my fable, the problem of human numbers in their relation to natural resources had been effectively solved. An optimum figure for world population had been calculated and numbers were maintained at this figure (a little under two billion, if I remember rightly) generation after generation. In the real contemporary world, the population problem has not been solved. On the contrary it is becoming graver and more formidable with every passing year. It is against this grim biological background that all the political, economic, cultural and psychological dramas of our time are being played out.

As the twentieth century wears on, as the new billions are added to the existing billions (there will be more than five and a half billion of us by the time my granddaughter is fifty), this biological background will advance, ever more insistently, ever more menacingly, toward the front and center of the historical stage. The problem of rapidly increasing numbers in relation to natural resources, to social stability and to the well-being of individuals—this is now the central problem of mankind; and it will remain the central problem certainly for another century, and perhaps for several centuries thereafter.

A new age is supposed to have begun on October 4, 1957. But actually, in the present context, all our exuberant post-Sputnik talk is irrelevant and even nonsensical. So far as the masses of mankind are concerned, the coming time will not be the Space Age; it will be the Age of Over-population."

- Aldous Huxley, Brave New World Revisited

The Age of Over-population

When highlighting the challenges to humanity and individual freedom that are presented in the various pathways to a Brave New World, Huxley argues that the straightest and the broadest of them is the road we are traveling today, the road that leads through gigantic numbers and accelerating increases. This will be applied to trends of organization and digitalization as we move forward, but Huxley places the blame for the central problem of mankind, as the problem of rapidly increasing numbers in relation to natural resources, to social stability, and to the well-being of individuals, that is, over-population.

Huxley's argument is that as societies (global, regional, and national) experience rapid increases in population growth, these rapid increases strain the economic and cult...
RFC WWIII
by Adam Zerner — 2022-03-01T09:10:18.441Z

You should ignore the news unless it's of historic import. Russia's invasion of Ukraine constitutes an event of historic import.

— Russia has Invaded Ukraine

One could argue for an even stronger position: you should ignore the news unless it 1) affects you and 2) there is something that you could do about it. I'm trying to think about whether 1 and 2 are true. Like most of us, I have some thoughts, but ultimately I'm not a geopolitics person and don't really know what I'm talking about. And so, this post is a request for comments[1], not an authoritative write-up.

May as well start now

Suppose we have the best case scenario: the war ends, tensions disappear, and we all go back to our lives. How long will that last? How long until tensions get serious again? 1 year? 5? 10? 25? 50? 100?

I lean towards the earlier end of that spectrum. 80,000 Hours would "guess the chance of a nuclear war is 2-20% in the next 200 years". Using that as a jumping off point, the chance of tensions developing enough where the threat is notable seems like it'd be a lot higher, especially given the current invasion.

Even if your estimate is towards the later end of this spectrum, it still seems like the sort of thing we will need to deal with in our lifetimes at some point. So then, any effort spent educating oneself and preparing right now probably won't be wasted. It's like buying honey for your pantry: it doesn't expire, and you know you will use eventually.

Decide what level of risk you are ok with ahead of time

There has been some talk about this in the context of covid. That you should decide ahead of time at what level of case counts you'd be ok with returning to eg. outdoor dining? What level for indoor dining? What level for large indoor gatherings?

Because if you don't, you risk some sort of status quo bias. And similarly, you risk the boiling frog thing happening to you. The case counts just slowly get lower and lower and lower, but each change is too small/gradual to get you to take action.

I think a similar principle makes sense in the context of nuclear risk. I could see tensions escalating, and escalating, and escalating, and escalating, but each escalation seems too small to justify an action like moving from the city to the country. To guard against this, I think it'd make sense to have some, preferably quantitative, idea in your mind about when you would take various a...
Seek Mistakes in the Space Between Math and Reality
by aphyer — 2022-03-01T05:58:15.419Z

Epistemic status: Cranky grumbling about the kids these days and all of their math and how they are using it wrong

Note: I'll be using a few examples as foils below. This isn't intended as criticism of those foils, it's because they provide good examples of what I think is a broader issue to be aware of.

Girl, I knew you were heaven-sent

When I'd explored the mate-space thirty-seven percent

Call off the search, come home with me

You're the highest-rated option after one-over-e

(SMBC).

The last plane trip of Keltham's first life starts out uneventful. He boards the aircraft, strolls a third of the way down the aisle with his eyes assessing all he passes, and then sits next to the first person who looks like a more promising seat-partner than all of the previous people he passed.

(mad investor chaos and the woman of Asmodeus, first lines).

The Secretary Problem is a famous result in mathematics. If you want to select the best option from a set, and you are presented these options one at a time and must choose either to accept or to reject, your optimal strategy is to auto-reject the first 1/e (about 37%) of options, and then accept the first option you see that is better than all previous options.

This is obviously a valuable real-world strategy in various fields, such as dating. As you can see, it has been mathematically proven that the correct strategy is to spend 37% of your dating life rejecting anyone you see, no matter how desirable they may appear, and then accept people you meet thereafter only if they are more desirable than anyone you saw before. If you are currently in a relationship, and are young enough to have less than a third of your adult life spent, you should immediately break up with your partner.

What? What do you mean, that doesn't sound right? It's been mathematically proven! Surely you don't think mathematicians have gotten this famous result wrong? Show me the flaw in the proof, then!

You can't find a flaw? Well, in that case, I reiterate, it has been proven mathematically that marrying your college sweetheart is a foolish idea, that there exists a superior strategy. Sadly many foolish people, perhaps as a result of insufficient math education, implement such mathematically suboptimal strategies, claiming to find 'true love' even though they objectively reduce their odds...
Late 2021 MIRI Conversations: AMA / Discussion
by Rob Bensinger — 2022-02-28T20:03:05.318Z

With the release of Rohin Shah and Eliezer Yudkowsky's conversation, the Late 2021 MIRI Conversations sequence is now complete.

This post is intended as a generalized comment section for discussing the whole sequence, now that it's finished. Feel free to:

raise any topics that seem relevant
signal-boost particular excerpts or comments that deserve more attention
direct questions to participants

In particular, Eliezer Yudkowsky, Richard Ngo, Paul Christiano, Nate Soares, and Rohin Shah expressed active interest in receiving follow-up questions here. The Schelling time when they're likeliest to be answering questions is Wednesday March 2, though they may participate on other days too.
Mundane, sharp, crazy
by Nyeogmi — 2022-02-28T20:02:24.182Z

This is a post about evaluating possible experts in fields you're not in, where you are somewhat qualified to evaluate their claims.

I try to think along these lines when talking to other engineers about their jobs, but also when figuring out which of two people has got their story straight in online drama.

I frame my model using competence claims, but I think you can use it whenever people are talking about extraordinary events that they witnessed and you haven't, as long as you've got a mental model of what people are likely to lie or exaggerate about.

You can even use my lines of reasoning for legal content once you know what some of the common heterodox legal takes are. (For instance, "fire in a crowded theatre" is not current precedent, but many people act like it is.)

I think my rules of thumb are pretty good, but I really recommend doing diligent research before you boost anything or publicly take a side in anything! When you do that, people other than you have the potential to be hurt or misled.

OK, here's my idea

When someone claims to be fourish standard deviations off the mean -- 160 IQ, for instance -- they're probably lying or mistaken.

Most of the time, when someone says "I'm competent," you have to weigh the odds that they really are against the odds that they're completely wrong. Especially when it's a really tall claim!

Many statements aren't claims of competence per se, but they have strong implications about competence. If you're in France and some guy comes in from England and you ask, "how did you get here?" then four worlds are possible. I'll assign them arbitrary percentage probabilities to make my point:

(9%) They say "I swam," but they didn't
(1%) They say "I swam," and they did
(89%) They say "I didn't swim," and they didn't
(1%) They say "I didn't swim," and they did

Well, based on these numbers and Bayes' Law:

2% of the time, they swam
If they say they swam, then 10% of the time, they swam

(The numbers are arbitrary, but you get the same conclusions as long as you assume that swimming is rare and people rarely lie in a way that implies they're less competent than they actually are.)

So this is all me building up to a general observation, which is that certain claims -- usually claims that sound crazy -- are equally likely to come from a competent or an incompetent person, because they remove the middle.

For instance, in software, you're likel...
Being an individual alignment grantmaker
by A_donor — 2022-02-28T20:02:10.683Z

I am an earlyish crypto investor who has accumulated enough to be a mid-sized grantmaker, and I intend to donate most of my money over the next 5-10 years to try and increase the chances that humanity has a wonderful future. My best guess is that this is mostly decided by whether we pass the test of AI alignment, so that’s my primary focus.

AI alignment has lots of money flowing into it, with some major organizations not running fundraisers, Zvi characterizing SFF as having “

too much money

”, OpenPhil expanding its grantmaking for the cause, FTX setting themselves up as another major grantmaker, and

ACX reporting

the LTFF’s position as:

what actually happened was that the Long Term Future Fund approached me and said “we will fund every single good AI-related proposal you get, just hand them to us, you don’t have to worry about it”

So the challenge is to find high-value funding opportunities in a crowded space.

One option would be to trust that the LTFF or whichever organization I pick will do something useful with the money, and I think this is a perfectly valid default choice. However, I suspect that as the major grantmakers are well-funded, I have a specific comparative advantage over them in allocating my funds: I have much more time per unit money to assess, advise, and mentor my grantees. It helps that I have enough of an inside view of what kinds of things might be valuable that I have some hope of noticing gold when I strike it. Additionally, I can approach people who would not normally apply to a fund.

What is my grantmaking strategy?

First, I decided what parts of the cause to focus on. I’m most interested in supporting alignment infrastructure, because I feel relatively more qualified to judge the effectiveness of interventions to improve the funnel which takes in people who don’t know about alignment in one end, takes them through increasing levels of involvement, and (when successful) ends with people who make notable contributions. I’m also excited about funding frugal people to study or do research which seems potentially promising to my inside view.

Next, I increased my surface area with places which might have good giving opportunities by involving myself with many parts of the movement. This includes Rob Miles’s Discord, AI Safety Support’s Slack, in-person communities, EleutherAI, and the LW/EA investing Discord, where there are high concentrations...
Ukraine Post #1: Prediction Markets
by Zvi — 2022-02-28T19:20:01.042Z

I am working on seeing how comprehensively I can cover and discuss the war, but that takes time and the speed premium is super high. I decided to start here, and write quickly. Apologies in advance for any mistakes, oversights, dumbness, and so on.

While I strive to provide sufficient Covid-19 news that you need not check other sources (and I will continue to do that), I will not be making any such claims regarding Ukraine even if I make regular posts. Please do not rely on me for such news.

The goal of this first post is to start our bearings with what prediction markets are available, what we might learn from them, and what markets might make them more useful.

Metaculus

Unfortunately, prediction markets so far have let us down when we need them most.

That is because the real money markets have so far been mostly unwilling or unable to touch questions surrounding the war, so most (but not all) of the markets we have to work with are on Metaculus (or Manifold Markets but that’s a mess).

Metaculus has some useful questions being asked, but Metaculus works by aggregation of all predictions. Even if you ignore their other issues, when something happens, the market is simply not going to adjust quickly.

This proxy market in particular seems illustrative.

This is on some level a profoundly silly question. The answer is not automatically 33%. As time outside the window goes by the conditional probability goes up, as time inside the window goes by it goes down. There could easily be seasonal effects as well. If provocative actions especially wars are more likely to happen during better weather, chances of war might be higher during the summer. So I’d be inclined, absent the Ukraine situation, to put this at 40%-45%, although I have no idea what you would do with that information.

Except now. Now we have the Ukraine war. If the nukes do fly, it seems like that would mostly happen before summer. Things are moving quickly. Thus, to the extent that there is a non-trivial short-term probability of nuclear war, this number should be moving. Yet it isn’t, because there is insufficient incentive to get people to do things that would cause it to move.

We can of course attempt to correct for such bias. If we know that Metaculus markets universally ‘move too slow’ then by tracking their rates of change we can get at least some amount of useful info. We can also comb...