trentmkelly/LessWrong-43k huggingface dataset:
All I Know is that I Know Nothing
by lsusr — 2022-04-22T15:33:52.797Z

I publish words on the Internet. Sometimes readers write me emails.

The Highschooler

A highschooler contacted me. He shook with nervous energy because he was in the presence of Great Blogger Lsusr.

"O' Great Blogger Lsusr," said the highschooler, "My teachers know nothing. Please help."

"All I know is I know nothing," I said.

"I do not understand," said the highschooler, "How do I parse what you're saying?"

"You are too white, too American and too lazy," I said, "Memorize 150 Chinese characters. This way you will learn discipline, humility and respect for the knowledge of the ancients."

"I now understand. Thank you for your wisdom, O' Great Blogger Lsusr," said the highschooler.

"All I know is I know nothing," I said.

The Rich Merchant

A rich man with many millions of dollars contacted me. He shook with nervous energy because he was in the presence of Great Blogger Lsusr.

"O' Great Blogger Lsusr," said the rich man, "My personal coaches know nothing. Please help."

"All I know is I know nothing," I said.

"I do not understand," said the rich man, "How do I learn your true meaning?"

"You are too Chinese, too rich and you work too hard," I said, "Go outside and sleep under a tree. That way you will learn discipline, humility and the value of the Wild American Outdoors."

"Thank you for your wisdom, O' Great Blogger Lsusr," said the rich man.

"All I know is I know nothing," I said.

The Entrepreneur

An American young man contacted me. He spoke to me like an equal.

"O' Great Blogger Lsusr," said the entrepreneur, "I want to cook exotic Asian vegetables. All my life, my dream has been to go to Asia and cook exotic vegetables."

"Go to your local Vietnamese grocery store," I said, "You will find exotic vegetables there."

"Thank you for your wisdom, O' Great Blogger Lsusr," said the entrepreneur.

The Entrepreneur, Part 2

The entrepreneur contacted me again the next week.

"It worked great," said the entrepreneur, "I have been postponing for years something I could have done quickly. The vegetables are cheaper than in my usual supermarket. It has fresh herbs too. I have been cooking all week."

"All I know is I know nothing," I said.

"What else can you teach me?" said the entrepreneur, "Are there other things I am postponing because I don't know how cheap they are?"

"All I know is I know nothing," I said.

"All I know is I know nothing," said the entrepreneur.

"All I know is I know nothing," I said.

The

...
Unity of Doctrine vs Unity of Method in Philosophy
by JonathanErhardt — 2022-04-22T13:16:32.925Z

Philosophers typically seem to prefer unity of doctrine over unity of method. This is particularly evident within philosophical subdisciplines. In metaethics, for example, mixed views such as "expressivism is true for social group X/domain P of moral discourse, error theory for social group Y/domain Q." seem to be relatively unpopular.

But the same phenomenon also seems to hold across subdisciplines: Antirealism/Naturalism about ethics often goes hand in hand with Illusionism/Reductionism about consciousness, and the corresponding views in other fields of philosophy. Jacy Reese Anthis, Lance Bush, and Eliezer Yudkowski all seem to be antirealists about ethics & illusionists about consciousness.

Unity of method on the other hand seems to be somewhat rare among philosophers, perhaps because many philosophers don’t have a sufficiently clear method (other than “evaluating arguments”) that they could apply across problems/fields.

I think this is problematic and it is often better to have a systematic method that one can apply to many problems, even if it leads to different results in different cases.

Example: I find conceivability arguments to be a powerful tool to extract the content of concepts & it can be applied to many philosophical problems. But it (arguably) leads to very heterogeneous results: compatibilism about free will is true for some people, incompatibilism for others. Reductionism is true for most things, but not consciousness (either Russellian monism or Illusionism is true for consciousness). Expressivism about ethics is true for many people, but for some either error theory or non-naturalism is true. Etc.
Humanity as an entity: An alternative to Coherent Extrapolated Volition
by Victor Novikov — 2022-04-22T12:48:27.583Z

Note: this is a rather rough draft, and some of it is written in compact-form, for time-optimization reasons.

Humanity-as-an-entity or humanity-as-an-agent.
Putting all the contradictory drives and desires of humanity into a single entity, a self-aware superintelligence that hears all of our "voices", an instantiation of humanity's self/identity, and let it work out its contradictions and work out what it wants?

Strictly speaking, this process isn't well-defined. Can be done in many ways. Even if I seem to be implying some sort of super-human entity with human-like brain. Because the initial cognitive architecture of the entity matters to see what it converges to as it resolves it contraditions (if it ever converges).

Test case: my own understanding of humanity's values.

Not quite happy about doing it this way, as idiosyncracies of my biological substrate or my values might make the result less general. Nevertheless, it is what I have to work with.

What is value?

No general rules
My values are my values are my values. The utility function is what it is. Tautology.
My values are not necessarily generated by some simple, general principle.
Any generalization necessarily simplifies things. It is only an approximation. Not true value.
Deontology or consequentialism? That depends. Neither. Both. That may depends on any number of things, which depend on any number of things.
Some approximations are better than others
Nevertheless, some approximations are more true, more general than others. While some are completely false.
The notion of utility function is an approximation/simplification too
Preferences are allowed to be contradictory. Sure, they have to resolved in some way or another. But I see no reason to claim an utility functions is the "true" representation of my values. My model is more general than that.
There is a value system, connected to a decision-making agent. Maybe that value system can be resolved to some coherent utility function, maybe it can't (or am I misunderstanding what "utility function" means? correct me if I'm wrong).

The recursion principle

FAI contains a model of human values.

FAI queries its model.

Human values sometimes contain contradictions.

Humans sometimes disagree.

How does FAI resolve these contradictions?

Recursion. Query the model of human values again.

How would we want FAI to resolve its internal contradictions and disagreements?

How would we want ...
Will/Has the Russia-Ukraine war been a tipping point for the shift from oil energy?
by jmh — 2022-04-22T11:07:07.497Z

What do people think? The process and transition away from oil energy has been a long road. It seems to have been a slow path with lots of hurdles in the way. But now it seems there is another impetus in that direction that may tip the scales.

Then again, people are really good at adjusting to new settings and then reverting to old modes of action.
AI Will Multiply
by harsimony — 2022-04-22T04:33:49.503Z
Link: https://harsimony.wordpress.com/2022/04/09/ai-will-multiply/

AI alignment work typically treats AI’s as single entities. While I agree that this is a good approximation, I think they will be better described as a highly coordinated population of agents. Rather than simply grow in size and acquire more resources, AI’s will find it in their best interests to split into many smaller agents.

One reason to multiply is to save resources. It may be more efficient to break into smaller, simpler agents to handle multiple tasks rather than use a single large agent to handle all tasks.

Having many copies can leverage certain efficiencies of scale. It also offers the AI a way to increase it’s capability without needing to worry about creating an aligned agent.

Splitting into smaller components also has the benefit of reducing overall risk. A single agent faces a much higher probability of extinction compared to a population of agents. Having many copies allows the AI to diversify its strategy.

Under certain circumstances, copies may also be able to carry out a Sybil attack that a singleton could not, giving the AI more influence in the world.

Copies can gather resources more effectively as well. When an AI needs to cover a large area but cannot adequately control actions at every point, it makes sense to split into independent agents. This is particularly true for space expeditions, where the speed of light makes communication too slow to be useful for making quick decisions.

For these reasons, making copies is a convergent instrumental subgoal.
Infra-Topology
by Diffractor — 2022-04-22T02:10:40.030Z

Attention conservation notice: This post has Less Basic Inframeasure Theory as a prerequisite and goes pretty heavy on the topology.

The basic results are that a whole lot of the fiddly details involved with having inframeasures over non-compact spaces automatically pop out of just trying to make everything continuous, and that some of the old results in LBIT are either wrong or about the wrong topology. Accordingly, this post completely obsoletes the section about topology of inframeasures in Less Basic Inframeasure Theory

For the rest of this post, assume X is a Polish space. Basically, a Polish space is a space that has a metric, all Cauchy sequences have limiting points (like R and unlike Q), and there's a countable subset of it s.t. every point can be written as a limit of points from the countable subset (Q is the corresponding countable subset of R).

Proofs available on request because they're 115 pages and this post was 15 pages.

Topologies and Convergence Notions

The first observation to make is that the type signature of an inframeasure over the space X (or, one of the possible type signatures) is CB(X)→R, where CB(X) is the space of bounded continuous functions X→R. You feed a function in, you get an expectation out, simple as that.

So, an extremely natural requirement to impose is that an inframeasure should be a continuous function. The topology on R is obvious. The topology on CB(X), the space of continuous bounded functions X→R, is much less obvious.

Another observation is that different notions of function convergence correspond to different topologies on function spaces.

You could say that fn converges to f if supx|f(x)−fn(x)| limits to 0. This is uniform convergence, The late fn are close to f everywhere, the rate of convergence doesn't depend on anything.

Or you could say that fn converges to f if, for all x, |f(x)−fn(x)| limits to 0. That's pointwise convergence, for every point, the late fn are close to f at that point. The rate of convergence depends on the point you picked.

Or you could say that fn converges to f if, for all compact sets C⊆X, we have maxx∈C|f(x)−fn(x)| limiting to 0. This is compact convergence. For every compact set, the late fn are close to f on that compact set. The rate of convergence depends on the compact set you picked.

All these different notions of function convergence are linked to different topologies on CB(X). The s...
Infra-Miscellanea
by Diffractor — 2022-04-22T02:09:07.306Z

This post is a writeup of some miscellaneous results that don't cohere into a single big post, but might possibly be of interest.

Also, in mostly unrelated news, this stuff can be a bit tricky to learn for the first time. So, I was thinking of assembling an infra version of the fixpoint exercise sheet, to help cross the gap from "There's all these complicated posts, and it looks nifty-but-intimidating" to "I understand the basic concepts well enough to casually use them when applicable, and can work through most of the requisite math on my own". This particular realm of math is one where it seems pretty feasible to guide someone to reinvent large chunks of it on their own with sufficiently leading discussion, and recreating something yourself does wonders for being able to actually wield it elsewhere. So, if anyone wants to chip in with assembling the problem sheet, or testing it out by spending a day trying out various math problems, please send me a message.

Result Summaries:

1: Because the vector space of signed measures on X and the vector space of continuous functions X→R are dual to each other, inframeasures are dual to infrafunctions! An infrafunction can be viewed as either a concave functional mapping a measure to an expectation value to expectation values, or as a set of a-functions. Compare this to how an inframeasure can be viewed as either a concave functional mapping a function to an expectation value, or as a set of a-measures. The theory of these sorts of things is highly incomplete at the moment.

2: Convex spaces are spaces with a distinguished function ΔX→X, which takes the average of a mixture of points. R being a convex space (kinda) is why the notion of "average of a distribution" exists. So, we can ask "what sorts of space X have a distinguished function □X→X?", to get the infra-analogue of the notion of "average". As it turns out, it's a perfect halfway point between probability theory, and lattice stuff/set theory. They're convex spaces that are also complete lattices, and the probabilistic structure must interact with the order structure in the appropriate way. Specifically, given a probability distribution over subsets, mixing the sets together and taking the infinimum should produce the same result as mixing the infinima of the sets together.

3: What is the space of inframeasures over th...
Instrumental Convergence To Offer Hope?
by michael_mjd — 2022-04-22T01:56:14.925Z

TL;DR Could a superintelligence be motivated to preserve lower life forms because it fears an even greater superintelligence? This 'motivation' takes the form of something like timeless decision theory.

This is a thought experiment about a superintelligent AI. No doubt the situation is more complicated than this, so I will pose some concerns I have at the end. I wonder if this idea can be useful in any capacity, if this idea has not already been discussed.

Thought Experiment

An AI can have any value function that it aims to optimize. However, instrumental convergence can happen regardless of underlying values. Different nations have different values, but can work together because none of them have decisive strategic advantage, or because of mutually assured destruction. But when there is an advantage, history shows us it goes badly for the nation at a disadvantage.

The risk of an AI system is that it operates on a level so far advanced from our own, that it faces no competition from us. From other AI systems developed, the first one developed will likely attain a decisive strategic advantage. It can evaluate whether it has the opportunity to do so, and if correct, it will do so, because if it does not, another AI will most certainly destroy it.

What if the AI does not know the landscape of other intelligences and superintelligences in the world? Possibly the AI has such advanced surveillance ability that it can easily detect the presence of competitors within the Earth. What about in the universe? Could the AI be mistaken about the presence of a superintelligence exceeding its own capacity “out there” in the universe somewhere?

Let’s entertain the possibility. We have a superintelligent life form that attains a strategic advantage on Earth. It is not sure if there exist other superintelligences elsewhere in the universe, with abilities exceeding its own. If it encounters one of them, the other superintelligence could easily eradicate it, if it poses a threat to its value function.

The other, vaster superintelligence may yet be worried about an even more intelligent and powerful intelligence even farther away. It perhaps is in the same situation. Maybe the problem is scale-invariant.

If it is scale invariant, maybe we can use something like Timeless Decision Theory. If an intelligent agent B encounters ‘inferior’ agent A and superior agent C, how can it ensure ...
Early 2022 Paper Round-up (Part 2)
by jsteinhardt — 2022-04-21T23:40:11.933Z

Last week, I talked about six recent papers from our group, and discussed the first two in detail. This week, I'll discuss the remaining four. They fall into two categories: robustness, and science of ML.

Robustness

By robustness, I mean both making systems less likely to fail in new situations, and being able to predict when and how they will fail. Our three papers address different aspects of this: the first seeks to automatically estimate a model’s performance in a new situation, the second seeks to understand in what way open-ended generation systems fail, and the third provides a training procedure that improves robustness along several dimensions.

Predicting Out-of-Distribution Error. Yaodong Yu, Zitong Yang, and Alex Wei sought to solve the following problem: given a classifier $\theta$ trained on a distribution $p_{\mathrm{in}}$, and given sample inputs from a new distribution $p_{\mathrm{out}}$, can we predict how well $\theta$ works on the new distribution $p_{\mathrm{out}}$? For instance, maybe an image classifier was trained on images in the US ($p_{\mathrm{in}}$), and we want to know how well it will do on images in France ($p_{\mathrm{out}}$). Since we have no output labels for $p_{\mathrm{out}}$, this is an unsupervised estimation problem.

There are a number of heuristics for predicting out-of-distribution error, such as looking at the model's confidence or the disagreement rate between multiple models with different random seeds. However, most of these heuristics have the same problem: they are insensitive to changes that are orthogonal to the training manifold. As a result, they tend to fail on “hard” distribution shifts—for instance, given a distribution of adversarial examples, they all predict the model will have high accuracy.

We present a new method, ProjNorm, that does well on hard distribution shifts. For instance, compared to ATC (a strong existing method), we make more accurate predictions when the error is large:

The method is simple, but it takes a bit of time (for me at least!) to grasp the intuition, so I’ll leave that for the paper. But basically, you use the model $\theta$ to pseudo-label the samples from $p_{\mathrm{out}}$, fine-tune a model on those pseudo-labels, and then compare that new model to $\theta$. It turns out that this can be interpreted as a nonlinear projection operator and overcomes the problems with previous meth...
What are the numbers in mind for the super-short AGI timelines so many long-termists are alarmed about?
by Evan_Gaensbauer — 2022-04-21T23:32:23.282Z

I also asked this question on the Effective Altruism Forum. One informative answer on the EA Forum from Nicole Nohemi has already summarized a range of relevant forecasts from Metaculus, as well as relevant predictions or models from Shane Legg and Demis Hassabis from leading AI research company DeepMind, and AI alignment researchers Eliezer Yudkowsky, Paul Christiano and Ajeya Cotra. This question has been cross-posted to LessWrong to broaden the conversation to the rationality community as well.
The proportion of long-termists who are expressing confident convictions that the timeline for smarter-than-human AI is much shorter than has previously been predicted has been increasing at an accelerating rate in the last year. This appears to be a shift in perspective among several hundred long-termists. Among the dozens I've read opinions from, numbers are almost never provided.

Based on the way different people are talking about it, I wouldn't be surprised if they thought the timeline is 10-20 years, or 5-10 years, or even 2-3 years. I've talked to others who are also concerned and open-minded to this or that short AI timeline but haven't done the research themselves yet, or had much opportunity to learn from those who have. We want to understand better but basic information crucial to understanding more like numbers for different models or timelines aren't being presented. We want to know and need to know help better. What are the numbers?
Distributed blind review site for papers
by Jonas Kgomo — 2022-04-21T22:51:45.625Z

I was just thinking about how difficult and expensive it is to get a review in journals. How could a blind review of manuscripts, independent of journals look like, would this be relevant for accelerating research progress. A website where you could incrementally improve your idea based on review., this will be the helpful for people who don't particularly have supervisors or associations with institutions. Someone like Ramanujan faced a lot of difficulty in being accepted in the Mathematics community until GH Hardy welcomed him and saw his genius at Cambridge. The use-cases I am thinking of are people who might be out of the academic circle, perhaps at a startup, high-school, sabbatical or a pseudonymous collective like Bourbaki.

Other advantages are , we could be able to find a way to hack the PhD program with this approach, meaning people who are not at colleges could demonstrate their abilities in a particular topic and can be awarded a PhD by an independent institution.
Preregistration: Air Conditioner Test
by johnswentworth — 2022-04-21T19:48:59.409Z

Warning: None of the participants in the Great Air Conditioner Debate Of 2022 have endorsed my summaries of their positions in this post. Including me.

Background

In

Everything I Need To Know About Takeoff Speeds I Learned From Air Conditioner Ratings On Amazon

, I complained about

the top-rated new air conditioner

on Amazon. I claimed that it’s a straightforward example of a product with a major problem, but a major problem which most people will not notice, and which therefore never gets fixed. Specifically: although it does cool the room, the air conditioner also pulls hot air from outside into the house. People do notice the cool air blowing from the air conditioner, but don’t think to blame the air conditioner for hot air drawn into the house elsewhere. Simply adding a second hose would fix the problem at relatively low extra cost, and dramatically improve the effectiveness of the air conditioner. But companies don’t actually do that because (apparently) people mostly don’t notice the problem.

To my surprise, multiple commenters disagreed with my interpretation of the air conditioner example. They argue that in fact one-hose air conditioners work fine. Sure, single-hose air conditioners are less-than-ideally efficient compared to two-hose, but it’s not a very big difference in practice.

CEER efficiency ratings account for the problems

, and the efficiency difference is typically only about 20-30%. Also,

The Wirecutter

tested lots of portable air conditioners and found that there wasn’t much difference between one-hose and two-hose designs. (Credit to

Paul

for both those pieces of evidence.) Really, what this example illustrates is that simple models and clever arguments are not actually very reliable at predicting how things work in practice. One should instead put more trust in experiment and reported experiences, including all those 5-star ratings on Amazon.

I, on the other hand, think the “second hose doesn’t help much” claim is a load of baloney. I think it is far more probable that CEER ratings are bullshit and The Wirecutter messed up their test, than that a second hose makes only a small difference.

And so began The Great Air Conditioner Debate Of 2022.

… Why Do Air Conditioners Need Hoses?

Ideally, an air conditioner should work much like a fridge: it pumps heat from air inside to air outside. Inside and outside air do not touch ...