trentmkelly/LessWrong-43k huggingface dataset:
Where are intentions to be found?
by Alex Flint — 2021-04-21T00:51:50.957Z

This is independent research. To make it possible for me to continue writing posts like this, please consider supporting me.

As we build powerful AI systems, we want to ensure that they are broadly beneficial. Pinning down exactly what it means to be broadly and truly beneficial in an explicit, philosophical sense appears exceptionally daunting, so we would like to build AI systems that are, in fact, broadly and truly beneficial, but without explicitly answering seemingly-intractable philosophical problems.

One approach to doing this is to build AI systems that discover what to do by examining or interacting with humans. The hope is that AI systems can help us not just with the problem of taking actions in service of a goal, but also with the problem of working out what the goal ought to be.

Inverse reinforcement learning is a classical example of this paradigm. Under inverse reinforcement learning, an AI observes a human taking actions, then looks for an explanation of those actions in terms of a value function, then itself takes actions that optimize that value function.

We might ask why we would build an AI that acts in service of the same values that the human is already acting in service of. The most important answer in the context of advanced AI, it seems to me, is that AI systems are potentially much more powerful than humans, so we hope that AI systems will implement our values at a speed and scope that goes beyond what we are capable of on our own. For this reason, it is important that whatever it is that the AI extracts as it examines a human taking actions is trustworthy enough that if it were implemented faithfully by the AI then the world brought forth by the AI would be a good world.

Inverse reinforcement learning is just one version of what I will call extraction-oriented AI systems. An extraction-oriented AI system is one that examines some part of the world, then, based on what it finds there, takes actions that affect the whole world. Under classical inverse reinforcement learning the particular part of the world that gets examined is some action-taking entity such as a human, the particular extraction method is to model that entity as an agent and look for a value function that explains its behavior, and the particular way that the system acts upon this value function is, at least under classical AI paradigms, to itself take actions that opti...
Why don't we vaccinate people against smallpox any more?
by  — 2021-04-21T00:08:31.593Z

The eradication of smallpox in 1979 represented one of the greatest achievements of modern civilization. However, since then most countries have elected to stop vaccinating their populations against the disease. This seems like a very concerning vulnerability to me, with waning herd immunity due to more and more of the world's population being replaced by unvaccinated young people.

What if there was an unintentional release from one of the labs around the world that still hold on to samples of the virus? What about an intentional release by terrorists/rogue nations? What gives scientists the confidence that there are no undiscovered animal reservoirs or uncontacted tribes in remote places where smallpox is still circulating? Smallpox is highly contagious and hundreds of times more deadly the SARS-Cov-2.

How would the world respond to such a release? Is there enough capacity to rapidly produce and deploy billions of doses of smallpox vaccines? (Right now we're at an all-time high in terms of pandemic preparedness; I'm thinking decades down the road when the lessons from Covid-19 has been all but forgotten)
What am I fighting for? 
by Quinn — 2021-04-20T23:27:26.416Z

Status: toward the end of writing this I started reading Suffering-Focused Ethics by Magnus Vinding as well as more Brian Tomasik, and I'm feeling myself value-drift in a more negative direction. It's possible I will endorse none of what follows fairly soon.

If you want to link to the higher-order freedoms formalization separate from the context of this post, just message me and I'll set it up in it's own post

Special thanks to comments from Nick Ronkin, Nicholas Turner

Motivation

It recently occurred to me that I can't be expected to calculate my behavior if I didn't put in the work of figuring out what I'm fighting for and performing backward induction.

Another word for figuring out what I'm fighting for is articulating my terminal values, which practically I think looks like painting dreams and hopes of what I think sentience ought to do. I will call this a specific notion of winning (SNoW). Crucially, though I'm seeking more detail I already know that an instrumental value is the world not dying, the great sentience project (on earth) not failing, etc. An argument against writing this post is the following: why be self-indulgent, writing amateur philosophy/scifi, when you've already converged upon a (n intermediary) goal that implies several behaviors?

You can imagine in the picture amputating the SNoW node and working entirely from the middle node! Just as, in a sense, the limits of my abstraction or foresight amputated an even further-right node in the first place (if you believe that no value is really terminal). However, I think there is a property inherent to backward induction that isn't captured by the syntax, the idea that as you move from right to left you're adding detail and motivation, every arrow brings complexity to its head node.

There is also to consider the colleague property of this setup: having a transhumanist/utopian terminal value endows me with the instrumental value of everything not blowing up, which endows me with the privilege of calling many of you my colleagues.

Indeed, it would not be shocking if Alice and Bob, having abolished x-risk and secured the future, were then at eachothers' throats because they disagreed so fundamentally about where to go from there. Natural questions arise: are they colleagues in the face of x-risk? Ought they be? The latter I will not attempt to answer, but my feeling about the former is that the answer...
Gradations of Inner Alignment Obstacles
by abramdemski — 2021-04-20T22:18:18.394Z

The existing definitions of deception, inner optimizer, and some other terms tend to strike me as "stronger than necessary" depending on the context. If weaker definitions are similarly problematic, this means we need stronger methods to prevent them! I illustrate this and make some related (probably contentious) claims.

Summary of contentious claims to follow:

The most useful definition of "mesa-optimizer" doesn't require them to perform explicit search, contrary to the current standard.
Success at aligning narrowly superhuman models might be bad news.
Some versions of the lottery ticket hypothesis seem to imply that randomly initialized networks already contain deceptive agents.

It's possible I've shoved too many things into one post. Sorry.

Inner Optimization

The standard definition of "inner optimizer" refers to something which carries out explicit search, in service of some objective. It's not clear to me whether/when we should focus that narrowly. Here are some other definitions of "inner optimizer" which I sometimes think about.

Mesa-Control

I've previously written about the idea of distinguishing mesa-search vs mesa-control:

Mesa-searchers implement an internal optimization algorithm, such as a planning algorithm, to help them achieve an objective -- this is the definition of "mesa-optimizer"/"inner optimizer" I think of as standard.
Mesa-controller refers to any effective strategies, including mesa-searchers but also "dumber" strategies which nonetheless effectively steer toward an objective. For example, thermostat-like strategies, or strategies which have simply memorized a number of effective interventions.

Richard Ngo points out that this definition is rather all-encompassing, since it includes any highly competent policy. Adam Shimi suggests that we think of inner optimizers as goal-directed.
Considering these comments, I think I want to revise my definition of mesa-controller to include that it is not totally myopic in some sense. A highly competent Q&A policy, if totally myopic, is not systematically "steering the world" in a particular direction, even if misaligned.
However, I am not sure how I want to define "totally myopic" there. There may be several reasonable definitions.

I think mesa-control is thought of as a less concerning problem than mesa-search, primarily because: how would you even get severely misaligned mesa-controllers? For exampl...
Thiel on secrets and indefiniteness
by Rob Bensinger — 2021-04-20T21:59:35.792Z

Some excerpts from Peter Thiel's 2014 book Zero to One that I've repeatedly come back to over the years:

[...] Why has so much of our society come to believe that there are no hard secrets left? It might start with geography. There are no blank spaces left on the map anymore. If you grew up in the 18th century, there were still new places to go. After hearing tales of foreign adventure, you could become an explorer yourself. This was probably true up through the 19th and early 20th centuries; after that point photography from National Geographic showed every Westerner what even the most exotic, underexplored places on earth look like. Today, explorers are found mostly in history books and children’s tales. Parents don’t expect their kids to become explorers any more than they expect them to become pirates or sultans. Perhaps there are a few dozen uncontacted tribes somewhere deep in the Amazon, and we know there remains one last earthly frontier in the depths of the oceans. But the unknown seems less accessible than ever.

Along with the natural fact that physical frontiers have receded, four social trends have conspired to root out belief in secrets. First is incrementalism. From an early age, we are taught that the right way to do things is to proceed one very small step at a time, day by day, grade by grade. If you overachieve and end up learning something that’s not on the test, you won’t receive credit for it. But in exchange for doing exactly what’s asked of you (and for doing it just a bit better than your peers), you’ll get an A. This process extends all the way up through the tenure track, which is why academics usually chase large numbers of trivial publications instead of new frontiers.

Second is risk aversion. People are scared of secrets because they are scared of being wrong. By definition, a secret hasn’t been vetted by the mainstream. If your goal is to never make a mistake in your life, you shouldn’t look for secrets. The prospect of being lonely but right—dedicating your life to something that no one else believes in—is already hard. The prospect of being lonely and wrong can be unbearable.

Third is complacency. Social elites have the most freedom and ability to explore new thinking, but they seem to believe in secrets the least. Why search for a new secret if you can comfortably collect rents on everything that has already been done? Every fall

...
Why “put down your phone and go outside” is Bad Advice. 
by madilynHex — 2021-04-20T19:05:11.424Z

If you’re like me in 2017, your phone is your drug of choice. My insurance didn’t provide me with any internet abuse rehab options. (Though at least one place exists.) I was sick of being a NEET and spending all my time online, so I gave my phone and laptop to a friend because a lot of people on Reddit had talked about this “cold turkey” approach. Cut to me asking for my devices back a day later. My friends were all too nice to actually hold me accountable. So I descended back into the murky ocean of online lurking, my life experience reduced to an endless scroll, searching for novel information and parasocial fantasies.

The cold turkey approach and similar “digital detox” methods are very popular in the “no surf” community. I see the appeal, it’s tempting to blame the internet on all of your problems and assume you’re better off without it. For hardcore internet users, which I believe is becoming the norm for young and old alike, it’s just probably not realistic or even all that helpful.

Cut to 2021- I’m in school and employed. I’m still an “internet person” but my phone screen time is around 2.5 hours per day when I used to hover around 15. I use my laptop for school and only fall into endless scrolling hell when I’m particularly depressed.

“Put down your phone and go outside.” didn’t work for me. I went to therapy. At some point I gained the ability to actually experience life directly, to be present in the current moment. I realized that the reason I had no life is because I was constantly escaping it. I had to start feeling my feelings. Running away wasn’t working.

That's the tldr; of how I escaped NEETdom. The recovery process is a bit more complex than I've alluded to, maybe I'll make a part 2 in the future.
Hard vs Soft in fields as attitudes towards model collision
by J Bostock — 2021-04-20T18:57:51.401Z

Many people will describe physics, chemistry, microeconomics and some parts of biology as "hard sciences" while describing psychology, sociology, politics and other parts of biology as "soft sciences". I think this taps into a set of attitudes within each field towards what we might refer to as model collision.

In fields which attempt to describe reality, different systems are described with different models. For example in physics we might use fluid dynamics to describe flowing water, rigid body mechanics to describe the movement of a set of gears, quantum electrodynamics to calculate the energy of a chemical bond between two hydrogen atoms etc. Each model can be considered to cover a certain area of reality, with different models covering different situations. Some models have clearly-delineated boundaries, sometimes there are gaps where nothing is understood, but in many cases it is not clear which model to apply. This could be because the boundaries of a model are not well defined within the model (what counts as a fluid?) or because two or more models overlap in scope (almost all economics and psychology). We can refer to this as a model collision.

For example (to use physics again) should we model not-quite-molten metal as a fluid which can flow or as a plastic solid body deforming under its own weight. In this case the answer is to experiment first and build a model later. In fields like this the collision of two different models can be resolved by experiment. It helps in physics that the underlying reality is well understood, in this case we know that the rigidity (or not) of a body is governed by the forces between particles.

Without the ability to experiment (or experiment reproducibly), as happens often in fields like psychology or politics, the two (or many more) models must end up coexisting. There are two ways for this to resolve: one is "everyone in the field puts appropriate weight on each model when making decisions, while searching for the truth" and the other is "the field splits into several angry mobs trying to prove that their model is the obviously correct one". Sadly the second case is more common (partially because it results in more papers being published). This can also result in models going in and out of fashion according to political concerns of those working to get grant funding. (I believe the hypothesis that protein aggregates c...
Young kids catching COVID: how much to worry?
by Steven Byrnes — 2021-04-20T18:03:32.499Z

Low confidence, slapdash job. Just putting this up in case other people want to compare notes.

(Update January 2022: Since writing this, I've adjusted my risk assumption down a bit based on newer studies suggesting lower rates of long-COVID than earlier studies (maybe a factor of 1.5 or something?). Then I adjusted down even more because it seems like omicron is less liable to cause serious illness than delta, holding everything else equal (maybe a factor of 2?). Granted there's more uncertainty about long-term effects of omicron, because it hasn't been around that long. But short-term effects have tended to correlate with long-term effects. Also, this is kinda unjustified based on my ignorance and lack of research, but in regards to permanent or long-term damage, I kinda have a good feeling about the fact that omicron is less likely to cause loss of taste and smell, it suggests to me that maybe it's more liable to leave nerve cells alone, and nerve cells are very important and (in some cases) bad at healing, cf. polio. (OTOH omicron seems to cause more brain fog, but that doesn't strike me as a nerve damage thing.) For the boosted adults and vaccinated kids in my family, I figure COVID is sufficiently low-risk that we should basically forget about it and live our lives. If I were the type of person to go bar-hopping, I would be bar-hopping. Except for one catch: our too-young-to-vaccinate kid. I'm figuring 99.5% chance that catching COVID wouldn't be a huge deal even for him. And avoiding catching COVID seems like it would require painful and damaging life disruption (e.g. pulling the kids out of school). So we're not doing that, and therefore we're figuring there's a very good chance we'll all catch it including him, and we're OK with that. We're still not indoor dining etc. during the current insane omicron peak (by contrast we were indoor dining just last month) because indoor dining is not something we care about very much, and because 60% chance of him catching it is still meaningfully better than 95% chance.)

(Update late February 2022: We're back to indoor dining!)

~~~

Now that all the adults in my family have been (at least partially) vaccinated, my kids will soon be the most COVID-vulnerable members of my family. Therefore it is newly decision-relevant to get a good sense for exactly how worried I should be about them catching COVID-19. This is April 2...
A Crackpot Physics: Issues
by ACrackedPot — 2021-04-20T17:07:52.150Z

An Introduction to A Crackpot Physics

An important part of developing a crackpot theory of physics is figuring out where the crackpot theory of physics deviates from normality.

Trying to elaborate on some of the bigger weaknesses in the model, I skip past some of the more mundane weaknesses; I could devote some time, for example, to noting the fact that the difference between light and gravitational waves in this model mostly come down to frequency, which is contradicted by the SM and spin mechanics. But, in another sense, that's just reiterating the idea that this isn't the Standard Model, so it isn't actually helpful to get a sense for where the ideas here have a shaky foundation.

Likewise, there's the central weakness of "I haven't actually identified the parameters that make my equations work at the scales under consideration", but again, the equation is not in fact the important thing in all of this.

Electrical Forces

The single biggest issue with the Crackpot Physics is electrical forces; for any of this to work, basically, electrical forces must be, in some sense, a fictitious force.

I've played with a few models to account for it. There is of course the Kaluza-Klein approach; that is one viable approach, although I think it isn't a fifth dimension at all, if Kaluza-Klein is the case, but rather time itself, which I believe might be a spiral in a complex plane. How a spiral related to a closed dimension is potentially complicated.

It could also be one of the two complex dimensions involved in the spiral which gives rise to the unified field; that is, a fictitious (in a sense) dimension. This has other interesting implications if correct, because the other complex dimension could potentially give rise to an anti-symmetric tensor.

However, my preferred explanation is that we don't need an explanation at all. Electron pressure is something like my preferred explanation. The basic idea here is that there is only the unified field theory. Protons repel one another in some range that I am reasonably certain can't go higher than 10^6 meters. They attract beyond that - gravity.

This predicts that magnetic fields should have a curious transition at some distance around 10^5, give or take an exponent, in which they briefly disappear, then reverse polarity. I've found limited evidence that some asteroid magnetic field...
Types of generalism
by Nora_Ammann — 2021-04-20T08:22:49.569Z

[Cross-posted from here]

I am interested in the nature of inter- and transdisciplinary research, which often involves some notion of “generalism”. There are different ways to further conceptualize generalism in this context.

First, a bit of terminology that I will rely on throughout this post: I call bodies of knowledge where insights are being drawn from “source domains”. The body of knowledge that is being informed by this approach is called the “target domain”.

Directionality of generalism

We can distinguish between

SFI

-style generalism from

FHI

-style generalism? (h/t particlemania for first formulating this idea)

In the case of SFI-style generalism, the source domain is fixed and they have a portfolio of target domains that may gain value from “export”.
In the case of FHI-style generalism, the target domain is fixed and the approach is to build a portfolio of diverse source expertise.

In the case of SFI, their source domain is the study of complex systems, which they apply to topics as varied as life and intelligence, cities, economics and institutions, opinion formation, etc.

In the case of FHI, the target domain is fixed, although more vaguely than it might be, via the problem of civilization-scale consequentialism and source domains include philosophy, international relations, machine learning and more.

Full vs partial generalism

Partial generalism: Any one actor should focus on one (or a similarly small number of) source domains to draw from.

Arguments:

Ability: Any one actor can only be well-positioned to work with a small number of source domains because doing this work well requires expertise with the source domain. Expertise takes time to develop, so naturally, the number of source domains a single person will be able to draw upon (with adequate epistemic rigor) is limited.
Increasing returns to depth: The deeper an actor’s expertise in two fields they are translating between, the higher the expected value of their work. This can apply to individual researchers as well as to a team/organization doing generalist researchers.

Full generalism: As long as you fix your target domain, an actor can and should venture into many source domains.

Arguments:

Ability: An actor can do high-quality research while drawing from a (relatively) large number of source domains, some of which they only learn about as they discover them. This

...
Does an app/group for personal forecasting exist?
by DirectedEvolution — 2021-04-20T05:04:01.418Z

I'm interested in personal forecasting - predicting my own future behavior on a range of timescales. I see it as a more useful skill than forecasting on world events. Formulating personally useful forecasts seems like an important and neglected skill in the rationalist community. And it would be nice to have some company, and tools to make it more convenient. Right now, I'm just using a spreadsheet. Does anybody know if there are groups doing this sort of thing? Is there a good app to manage the process?