trentmkelly/LessWrong-43k huggingface dataset:
Confidence intervals seem to be rarely useful, in and of themselves
by anorangicc — 2022-02-05T11:23:16.132Z

Context: I read Eliezer Yudkowski's post "Frequentist statistics are frequently subjective" and it inspired me to get a better first-principles understanding of some statistical concepts. Now these are my personal thoughts about the topic. Don't believe me; if I had attached confidence intervals to the results of my investigation, they would be very broad!

Confidence intervals frequently cause confusion. For example, the U.S. National Institutes of Health (NIH) claim in their course on "Finding and using health statistics":

Confidence intervals are frequently reported in scientific literature and indicate how close research results are to reality, or how reliable they are, based on statistical theory. The confidence interval uses the sample to estimate the interval of probable values of the population; the parameters of the population.

For example, if a study is 95% reliable, with a confidence interval of 47-53, that means if researchers did the same study over and over and over again with samples of the whole population, they would get results between 47 and 53 exactly 95% of the time. The reliability in this example refers to the consistency of the measurement, or the ability to repeat it. Poor reliability can happen with a small population, or if the health event being studied does not happen often or at regular times.

(emphasis added)

This paragraph left me confused but I believe the statement is wrong. To get to this point, let's first give a definition of a confidence interval.

Let's assume a situation where we want to determine some value q∈R telling us something important about the world (global mean temperature, ratio of defective items in a sample of products, ...). So we design some experiment resulting in a vector of measurable results X of random variables. The distribution of X depends in a assumed-to-be-known way on q.

Does the outcome of the experiment, the realization of X instead of its distribution, tell us something about p? Not with certainty. However, we can use strategies that will often succeed giving us true information about q.

When I say "confidence inverval" for q inferred from X, this isn't a canonically well-defined concept in and of itself. A method to assign confidence intervals is a function C that maps each realization of X to an interval of real numbers...
Real Reality
by ship_shlap — 2022-02-05T04:13:40.309Z

Disclaimer: This may not be a very new concept, but it provides a revisions or a slightly different perspective on the matter.

I realized that I started to talk less with my dad. It's because his hearing is poor (due to some ear infection he had in his childhood). It would be awkward whenever I would say something to him, and he didn't know that I was trying to say something, let alone hearing a blur of a sound from me so that he can ask me to reiterate what I said. It would be doubly awkward because he sits more often around family members, so everyone else would be there to witness it. It would take effort to repeat words to him, and then the awkwardness would increase and feed into a positive feedback loop the more I tried. Soon, due to talking less with my father, I lost the urge to share that 'cool fact I saw online' and 'Dad I saw something really cool'. This was another piece of good justification for not talking as much, realizing that the urges were there as a habit (which it essentially is). My mother later pointed out that I started to talk less to my father. I started making justifications in my head, such as the awkwardness, the effort, etc. I do love my father, but I rightly believed that all my reasons were justified for talking to him less. I was still open to other thoughts so I let the different what ifs trickle in. All of a sudden, I had this realization. It was cliché, but this time, I knew it was different. The shift I felt was of pure understanding, and it also felt like a shift in emotional understanding.

My fathers view/model of the world/reality simply held the fact that I talked to him a lot less. He couldn't see the internal struggle I experienced each time I had to talk to him. This is what my model of 'reality' looked like:

Figure A: My 'model of reality' for talking to my father less.

On the other hand, this was his model, which is drastically different:

Figure B: My father's model of reality, far more different than mine.

This whole difference in models reminds me of Aumann's Agreement Theorem where if we have the same priors, we should have the same posteriors (Ignoring the concept of agreeing to disagree). Our models of the world are so different: If we have very different perceptions, and more or less different perceptions, how can we find the correct answers to many problems we face? How can we hope to ever achieve the model of a...
Why will AI be dangerous?
by Legionnaire — 2022-02-04T23:41:36.810Z

LessWrong should offer a short pitch on why AI will be dangerous, and this aims to be that.1

Many people think, "Why would humanity make dangerous AI? Seems like a stupid idea. Can't we just make the safe kind?" No. Humanity will make dangerous AI for the same reason we made every other technology dangerous: it's more useful.

A knife sharp enough to cut fruit can cut your finger. Electrical outlets with enough power to run your refrigerator can stop your heart. A car with enough horsepower to carry your family up a hill can easily kill pedestrians. Useful systems must be dangerous because useful implies they can have large effects on their environment. "Large effects" can be good or bad depending on your perspective.

We'll make AI powerful too: knowledgeable enough to cure diseases, tactical enough to outsmart terrorists, and capable enough to run an economy by itself, all so we can relax! That means our AI will also be knowledgeable enough to invent new diseases, tactical enough to outsmart freedom fighters, and capable enough to run a military-industrial complex all by itself.

We won't do the latter on purpose any more than we crash cars or cut our fingers on purpose, the problem is that there are more ways for complex systems to go "wrong" than "right" because this universe is psychopathic, even when nobody is being malicious. The asteroid that killed the dinosaurs wasn't malicious, it just didn't care. Nowhere in the behavior of the asteroid was encoded a consideration of life, or even itself. It just followed mathematical laws (of gravity) which don't care about "morality".

Like the asteroid, AI systems just follow mathematical laws (of intelligence) which don't care about "morality". We'll add safety components that can consider moral questions, but mistakes will be made as have been made with all previous technologies.

The odds of a mistake go up a lot when you consider that the whole selling point of AI is to be smarter than us, and that entities smarter than you are unpredictable. They will do things you didn't think of, sometimes things you didn't even know were possible. AI presents a unique and unfamiliar danger to humanity, because unlike other technologies, an AI disaster might not wait around for you to come clean it up. That wouldn't be very intelligent, would it?

Footnotes

1. Most posts are long, detailed, and use esoteric language. Not g...
Anti-Parfit's Hitchhiker
by k64 — 2022-02-04T23:37:12.247Z

Thinking about Parfit's Hitchhiker, an alternative example occurred to me:
You're lost in the desert and this time Aul Peckman drives up and tells you "I will give you a ride back to town iff you would have stiffed my nemesis Paul Eckman." After reading Parfit's Hitchhiker, you had pre-committed to pay Paul Eckman if this happened to you, or chosen a decision theory that would cause you to do that, so you try telling Aul Peckman that you would stiff his nemesis in this situation, but he knows you're lying and drives off. If only you weren't so timelessly rational!
Obviously, one can argue that you're more likely to encounter agents who will want to get paid than who will want you to not pay someone, and so if you're in a world where that is true, you still have positive EV from running TDT/UDT, but is this an example of regretting TDT rationality?
Is it rational to modify one's utility function?
by k64 — 2022-02-04T23:37:12.177Z

Rationality is often informally defined by means-end reasoning or utility maximization. However, this idea becomes less clear when faced with the option of modifying one's own utility function. Does rationality prescribe avoiding any change to one's current utility function because such a change would obviously reduce expected utility under the current function, or does it prescribe taking actions which result in the highest utility by whatever means necessary, in which case a change would be rational iff the new utility function yields higher expected utility given known background info about the world?
This is obviously relevant to AI alignment, where one concern is that AI may hack their own utility functions and another concern is that they may prevent humans from modifying them (or shutting them off) due to the risk to their current goals. It's also relevant to questions of human rationality, where, on the one hand, we imagine that Ghandi would not take a pill that makes him want to murder people, but on the other hand, we regularly believe that unhappy people should change their own psychology and goals to be more happy.
Blog dedicated to rebuilding technology from civ collapse? 
by Casey_ — 2022-02-04T23:28:01.754Z

I'm trying to re-find a blog/website that had a prepared list (with pictures/descriptions) of all the basic technology/machines we would need to rebuild civilization. These machines altogether would cost tens/hundreds of thousands, but I'm imagining various communities would be able to chip-in for at least some of this gear to help distribute our redundant resilience in the case of civ collapse.
Alignment versus AI Alignment
by Alex Flint — 2022-02-04T22:59:09.794Z

Financial status: This work is supported by individual donors and a grant from LTFF.

Epistemic status: This post contains many inside-view stories about the difficulty of alignment.

Thanks to Adam Shimi, John Wentworth, and Rob Miles for comments on this essay.

What exactly is difficult about AI alignment that is not also difficult about alignment of governments, economies, companies, and other non-AI systems? Is it merely that the fast speed of AI makes the AI alignment problem quantitatively more acute than other alignment problems, or are there deeper qualitative differences? Is there a real connection between alignment of AI and non-AI systems at all?

In this essay we attempt to clarify which difficulties of AI alignment show up similarly in non-AI systems, and which do not. Our goal is to provide a frame for importing and exporting insights from and to other fields without losing sight of the difficulties of AI alignment that are unique. Clarifying which difficulties are shared should clarify the difficulties that are truly unusual about AI alignment.

We begin with a series of examples of aligning different kinds of systems, then we seek explanations for the relative difficulty of AI and non-AI alignment.

Alignment in general

In general, we take actions in the world in order to steer the future in a certain direction. One particular approach to steering the future is to take actions that influence the constitutions of some intelligent system in the world. A general property of intelligent systems seems to be that there are interventions one can execute on them that have robustly long-lasting effects, such as changing the genome of a bacterium, or the trade regulations of a market economy. These are the aspects of the respective intelligent systems that persist through time and dictate their equilibrium behavior. In contrast, although plucking a single hair from a human head or adding a single barrel of oil to a market does have an impact on the future, the self-correcting mechanisms of the respective intelligent systems negate rather than propagate such changes.

Furthermore, we will take alignment in general to be about utilizing such interventions on intelligent systems to realize our true terminal values. Therefore we will adopt the following working definition of alignment:

Successfully taking an action that steers the future in the direction of our tru

...
A Quick Look At 20% Time
by Elizabeth — 2022-02-04T22:10:01.672Z

I was approached by a client to research the concept of 20% time for engineers, and they graciously agreed to let me share my results. Because this work was tailored to the needs of a specific client, it may have gaps or assumptions that make it a bad 101 post, but in the expectation that it is more useful than not publishing at all, I would like to share it (with client permission).

Side project time, popularized as 20% time at Google, is a policy that allows employees to spend a set percentage of their time on a project of their choice, rather than one directed by management. In practice this can mean a lot of different things, ranging from “spend 20% of your time on whatever you want” to “sure, spend all the free time you want generating more IP for us, as long as your main project is completely unaffected” (often referred to as 120% time) to “theoretically you’re free to do whatever, but we’ve imposed so many restrictions that this means nothing”. I did a 4-hour survey to get a sense of what implementations were available and how they felt for workers.

A frustration here is that almost all of what I could find via Google searches were puff-pieces, anti-puff-pieces, and employees complaining on social media (and one academic article). The single best article I found came not through a Google search, but because I played D&D with the author 15 years ago and she saw me talking about this on Facebook. She can’t be the only one writing about 20% time in a thoughtful way and I’m mad that that writing has been crowded out by work that is, at best, repetitive, and at worst actively misleading.

There are enough anecdotal reports that I believe 20% time exists and is used to good effect by some employees at some companies (including Google) some of the time. The dearth of easily findable information on specific implementations, managerial approaches, trade-offs, etc, makes me downgrade my estimate of how often that happens, vs 20% time being a legible signal of an underlying attitude towards autonomy, or a dubious recruitment tool. I see a real market gap for someone to explain how to do 20% time well at companies of different sizes and product types.

But in the meantime, here’s the summary I gave my client. Reminder: this was originally intended for a high-context conversation with someone who was paying me by the hour, and as such is choppier, less n...
What is a “philosophy of progress?”
by jasoncrawford — 2022-02-04T22:02:55.926Z
Link: https://rootsofprogress.org/what-is-a-philosophy-of-progress

I’ve been using the term “philosophy of progress” a lot lately (such as calling for a new one, or critiquing the old ones). What does this term mean?

I use the term analogously to philosophy of science, philosophy of law, or philosophy of education. There are certain foundational questions relevant to the study of progress that border on or overlap with philosophy.

In outline, here are some of the main questions that I see as making up the philosophy of progress, grouped into four top-level topic areas: definition, evaluation, causation, and prescription.

Definition

What is progress?
What kinds of progress are there? (Scientific, technological, economic, moral, political…) How do they relate?
What is “true progress” or “human progress?”
How can we measure any of the above?

Evaluation

What are the benefits of (each type of) progress? Are they real or apparent? Healthy, or an “addiction?”
Specifically, how does progress relate to human happiness and well-being?
What are the costs and risks of progress? (Safety, economic upheaval, war, environmental impacts, health impacts, etc.?) Do they outweigh the benefits?
How do we make these tradeoffs?
Bottom line: is progress good?

Causation

What causes progress?
Regarding material progress specifically, what is the role of science? of economic freedom? of government investment? of individual geniuses or great leaders? of political stability? of corruption or lack thereof? of financial institutions? of natural resources? of societal trust? of other social norms? etc.
Are there inherent limits to progress?
What caused progress historically—why did it happen when and where it did?
What explains the pace of progress over time? On the face of it, progress seems to have been very slow for most of human history, and much faster in the last few centuries—why?
What can we expect for the pace of progress in the future? Will it continue, grind to a halt, accelerate to a singularity, something else?
How much control or agency do we have over progress?

Prescription

In the end, how should we regard progress and what should be our stance towards it?
Specifically, how should we regard inventors, scientists, entrepreneurs, policy-makers, social reformers, political activists, etc.? What social status should we grant them? Should we celebrate their achievements, and if so how?
How should we communicate about progress? How should it be taught in school? How shou

...
Convoy
by Zvi — 2022-02-04T21:40:02.393Z

This would have been in yesterday’s weekly post but I ran out of time, so I only got to it today. It seemed substantial enough to post on its own rather than wait a full six days, so here it is.

Reports are in from Canada. Looks like we got us a convoy. [NOTE: This post has been extensively edited to reflect that what previously was a convoy is now more like an occupation akin to the old ‘autonomous zone’ in Seattle, and conflicting reports about exactly what to make of these protesters commenters have shared with me.]

Many truckers do not like vaccine mandates.

This makes sense. The big benefits of being a trucker are:

Freedom.

In particular, freedom to go and work where you want, when you want.

Even more in particular, freedom to not interact with other humans.

If there’s one group that both has no need of vaccine mandates, since they spend their time alone, and also likely not to take kindly to vaccine mandates, it would be a trucker. As far as I can tell, the whole advantage of being a trucker, other than getting paid for doing something other people don’t want to do, and something something about the joys of the open road, is to have and frequently use full f*** you rights without the need for f*** you money.

Truckers are also both vital to the supply chain and therefore the economy while being in short supply, and also are mostly physically large people in possession of a large number of very large trucks. And those who don’t want to get vaccinated are facing a lot of restrictions that no longer make physical sense:

'I can travel freely through the border, and not be in contact with anyone. Yet I'm locked into my own country right now,' he said. 'I can't go on a holiday. I can't go to a restaurant, I can't go bowling. I can't go to a movie. You know, these are things that it's just gotten out of control.'

They also are not happy about no longer having an exemption to quarantines and tests for crossing the border if they’re not vaccinated, but that seems like it shouldn’t matter much in practice given the United States won’t let them in unvaccinated at all. Not that any of that makes any physical sense at this point, of course, it’s purely punishment.

It is not terribly surprising that they decided to let their discontent be known and started a convoy to head to Ottawa to peacefully protest, with lots of emphasis on how violence would be counterproductive. A...
Can you prove that 0 = 1?
by  — 2022-02-04T21:31:29.856Z

Of course, 0 is not equal to 1. But certain interpretations of quantum mechanics imply that a thing can be one way and be the opposite way at the same time. Any mathematical proof rests on axioms. What I'm looking for is a proof that 0 = 1 which rests on axioms that are being actively debated by mathematicians. (Then, I'll argue that the truth of 0 = 1 is in a quantum superposition with the debates being held by mathematicians.) Feel free to create your own proof, or to direct me to existing works. I may quote your answers in a soon-to-be-published post on this topic.

This doesn't feel frivolous to me; I think that 0 = 1 hints towards a unifying theory for epistemology, math, consciousness, states of enlightenment, and physics. It also has implications for AI. If anyone has meta-advice for me, I'm open to it. I think the benefits balance the negatives of sharing these ideas, especially for ensuring that they're debated more widely and openly. But I could be wrong. (There are keywords I'm not using to be careful.)
Expectations Influence Reality (and AI)
by  — 2022-02-04T21:31:29.718Z

Edit: I want to be very clear that I'm not espousing the law of attraction. Instead, I'm saying that expectations influence reality—but always indirectly, through our actions and behavior (as informed by our beliefs). The most accurate level of description will always follow the laws of physics, without any spooky direct influences from our beliefs or expectations.

Epistemic status: Exploratory. This idea is based, in part, on hard-to-explain personal experiences with enlightenment states. I know I can explain the underlying model that led me to this conclusion, but it will be time-consuming, and I wanted to publish some of the "implications" from the model first. This post may make more sense to people who have experienced enlightenment or are at least familiar with the three marks of existence.

I've been following LessWrong for about two years, which is just as long as I've been an effective altruist. This topic is weird for a first post; I hope you'll forgive me for honestly stating something that I feel is very important and neglected in this community: the language used to talk about AGI progress and the expectations evoked by that language—both in capabilities and in alignment theory. What expectations should we have about the pace of AGI progress? Is there a right way to feel? I don't think there is a universally true answer (devoid of any implicit framing), but I will argue for a pragmatic response. This also has "self-improvement" applications, although that's not my intention.

Expectations Influence Your Future

When you're trying to update your beliefs about the larger world to match reality, it's helpful to assume that your expectations play no role in how reality will turn out. If you assume otherwise and look for evidence to confirm your current expectations, you will have fallen prey to confirmation bias. That's all fine and good until reality slaps you in the face with disconfirming evidence. (I could link a hundred LessWrong articles here; these are not new ideas.) The lesson we're supposed to take away is that if our beliefs don't match reality, then our beliefs are the problem and must be shifted in the right direction.

However, there are occasions when it's more helpful to adjust reality first and let your beliefs follow suit after—namely, in your personal life.

Suppose that you are depressed and feel hopeless. Life seems awful, and things you p...