trentmkelly/LessWrong-43k huggingface dataset:
Clarifying the palatability theory of obesity
by Matthew Barnett — 2022-02-10T19:16:03.555Z

Here, I intend to clarify the core claims of the palatability theory of obesity outlined by Stephan J. Guyenet in his book The Hungry Brain (reviewed by Scott Alexander). Previously, I had written a comment defending his theory against a recently popular alternative, proposed by Slime Mold Time Mold (SMTM), which states that environmental contaminants are responsible for the ongoing obesity epidemic.

As with many other topics, I often find public debates about obesity frustrating because it's sometimes unclear to me what people mean when they champion a theory. To address this concern, I will briefly summarize (but not necessarily defend) what I see as the key elements of Guyenet's theory, including how it can be empirically falsified and distinguished from its competition.

The theory in a nutshell

The palatability theory is a theory of why obesity has increased over time. It posits that the primary cause is an increase in reward signals in the modern environment from processed food, mediated by the motivation system in the brain, causing us to overeat.

This sounds very simple, but there are many ways to be confused about what the theory is actually claiming, and indeed, I think many people are at present confused about it. Therefore, I'm going to go through the main parts of the theory, step by step, in the hopes that this clears up some confusion.

The datum to be explained

The palatability theory of obesity is mainly trying to explain why humans, especially those in rich nations like the United States, on average weigh more and have higher body fat percentages than their recent ancestors at similar ages.

Note that this is different from saying that the theory is trying to explain variation in individual obesity, at any given time, within a society. The palatability theory does not provide any direct explanation for why some people are more obese than others, whether and why some people's metabolisms are more efficient than others, why some people have more difficult time dieting than others, or why some people prefer hyperpalatable food relative to others.

To be sure, the palatability theory can shed light on each of these questions, and it can certainly be empirically tested using data collected across space rather than across time. For example, if we compared two contemporaneous communities, one whose members ate a modern "cafeteria diet" and another whose membe...
A summary of aligning narrowly superhuman models
by gugu — 2022-02-10T18:26:29.127Z

This post was written under Evan Hubinger’s direct guidance and mentorship, as a part of the

Stanford Existential Risks Institute ML Alignment Theory Scholars (MATS) program

. Where not indicated, ideas are summaries of the documents mentioned, not original contributions. I am thankful for the encouraging approach of the organizers of the program, especially Oliver Zhang.

TL;DR

In the post “

The case for aligning narrowly superhuman models

”, Ajeya Cotra advocates for performing technical alignment work on currently available large models using human feedback, in domains where (some) humans are already outperformed (in some aspects). The direct benefit of this line of work could be testing out conceptual work, highlighting previously unknown challenges and eliciting new scalable, general solutions, as well as indirectly moving the field of ML in a better direction and building the community and infrastructure/tooling of AI Safety. This post summarizes the original post, some related work and some objections.

Short intro

Progress in ML could allow us to do technical alignment research that is closer to the "real problem" than specific toy examples. In particular, one of the main promising directions (among others such as

Interpretability

and

Truthful and honest AI

) is discovering better methods of giving feedback to models more capable than us. This might be easy for some concrete, narrow tasks, such as playing Go (there is an algorithmic way to decide if a model is playing Go better than another model), but for more “fuzzy” tasks (such as “devise a fair economic policy”), we can’t evaluate the model according to such a gold standard.

A couple of examples of work falling into this category (See Example tasks):

Train a language model to accurately explain things about a field that the feedback providers are not familiar with.
Train an RL agent to act well in an environment where the RL agent can observe more information than the feedback providers can.
Train a multilingual model to translate between English and a foreign language that the feedback providers do not know. [Summary after

Alignment Newsletter

]

These tasks call for more sophisticated ways of generating feedback for the models. Some of the approaches that can be used include debate or decomposing the question into subtasks. See section Possible approaches to alignment.

To test these methods,...
Covid 2/10/22: Happy Birthday
by Zvi — 2022-02-10T17:50:02.081Z

Happy birthday. This week CNN’s home page literally led with a story about a boy trapped down a well, then pivoted to Olympic coverage with a side of continued complaining about Donald Trump. Two years into the pandemic, we have normality.

Well, almost. There’s still a bunch of unnecessary restrictions in place and various arguments about them. That is going to go on for a while. The BA.2 variant is going to take over, although I doubt it will cause major issues as it does. There’s also that whole convoy situation.

Still. With notably rare possible exceptions, there’s nothing we can’t cope with. For those worried about Long Covid, I wrote The Long Long Covid Post.

Thus, I’m going to take this week’s title to actually say: Happy birthday, dad.

Executive Summary

BA.2 will take over as dominant subvariant but likely changes little.
Deaths finally in decline.
Cases declining rapidly.

Let’s run the numbers.

The Numbers

Predictions

Prediction from last week: 1.49mm cases (-40%) and 18,000 deaths (+2%).

Results: 1.40mm cases (-43%) and 17,028 deaths (-3%).

Prediction for next week: 900k cases (-36%) and 14,700 deaths (-13%).

I see a few signs the decline in cases will slow as people adjust, including that the places in most rapid decline now have a much smaller share of remaining cases, and also there’s no reason to expect substantial speeding up from here so on average things will slow.

On deaths, we saw a slightly premature decline due to the Midwest peaking early. We’re looking at about a 20% decline in three-week-old cases, so we should see a modest decline here but the meteoric drops shouldn’t start until the week after next. Then again, we don’t know much about Omicron’s timing, so I am ready to be surprised.

Deaths

The Midwest dropping now seems early, which is likely why the number came in slightly low. The descent starts now. Once it does, the calls for lifting what restrictions remain should rapidly get louder.

Cases

In true case numbers news, here’s a study of unvaccinated measures how often they previously had Covid:

Of 1580 individuals invited to undergo serologic testing, 816 (52%) did so between September 24, 2021, and November 5, 2021. Participants had a mean age of 48.0 years, 421 (52%) were women, and 669 (82%) were White (Table). Fourteen percent reported routine mask use in public. Anti-RBD and anti-N

...
Question 1: Predicted architecture of AGI learning algorithm(s)
by Cameron Berg — 2022-02-10T17:22:24.087Z

Exploring the possible space of learning architectures

Here is the ultimate challenge: in order to control something, one generally has to understand how it works—i.e., to accurately predict how it will behave under various manipulations.

I sometimes have a recurring dream that I am aboard a plane falling out of the sky, and for whatever reason, I, a lowly passenger, am suddenly tasked with landing the plane safely. However, when I enter the cockpit, all I see are a series of levers, buttons, dials, dashboards, etc. Unsurprisingly, I do not understand the function of any of these mechanisms in my dream; I thus always fail my mission to properly fly the plane. It is crucial for existential risk minimization that we do a better job understanding and controlling the trajectory of AGI than my dream self does with flying a plane.

In order to control the behavior/consequences of the behavior of AGI, we will have to understand how it works to a sufficient degree.

In order to understand how AGI works, we will have to understand the general strategies that the AGI employs to achieve goals and solve problems.

And understanding these ‘general strategies’ is really a question of the architecture of AGI’s learning algorithm(s). This is because these learning algorithms constitute the specific mechanism by which the AGI would transition from the state of being unable to solve some problem/achieve some goal to being able to do so.

A quick note on the ‘(s)’ in learning algorithm(s): there is no reason for us to exclude from consideration the possibility that AGI will have multiple learning strategies at its disposal, employing its different learning algorithms for different problem-solving contexts. With this being said, I think it would still make sense in many cases to talk about the ‘fundamental’ or ‘foundational’ learning algorithm employed by the AGI. For example, if the AGI fundamentally represents the world as a vast set of states (perhaps using SL) inside of which it attempts to actuate sequences of high-value actions, this AGI’s ‘foundational’ learning algorithm would be RL (even though it also uses some SL).

I will now outline my best guess at the space of these possible learning algorithm architectures. I’ve divided up the space into three axes: foundational learning algorithm, post-deployment learning capacity, and level of general intelligence....
Rituals and symbolism
by TsviBT — 2022-02-10T16:00:14.635Z

[Context: In the seven years I've been living in the Bay Area, I've attended two Rationalist Solstices. Both times I basically hated the experience. I don't really know why, but in thinking about it I think I've understood something worth saying about rituals and symbolism. (I don't think what follows is a crux for me about Rationalist Solstice (I don't even know if it applies); I'd guess that has more to do with people gathering in large groups in general seeming harmful, distortive, or hostile or something.)]

On Friday evening--Erev Shabbat (= evening of the day of rest)--Jews light candles and say a brachah. I was told by adults that this is done to symbolize a separation between the week and the holy Shabbat. On Saturday evening, when Shabbat is over, Jews do Havdalah (= separating): they light another candle; smell spices; and say more brachot. I was told that the warmth and light of the candle serve to remember to us the warmth and light of Shabbat as we enter the week; and the spices, to carry the sweetness of Shabbat. On Sukkot, they build a sukkah (hut) with a roof of loose branches. That way you can see the stars and be open to the universe.

Nothing terribly wrong with all that, but it's not what the candles are for. Jews used to light candles on Erev Shabbat because they wanted to have light in the evening, and they couldn't (by the prohibitions of Shabbat) light candles after it got dark. They lit candles on Saturday evening for the same reason, just after Shabbat had ended. They ground or heated spices (and drank wine) as part of the evening meal just after Shabbat, and maybe then secondarily to show to people (create common knowledge) that the prohibition against grinding or heating things is lifted [1]. And the sukkah was a makeshift hut farmers used during harvest, maybe for shade, or to watch over vulnerable ripe crops at night.

A definition of ritual this suggests: A ritual is a form of an originally purely functional action, stereotyped and exaggerated to strengthen an effect on observers that the original action already had. Stereotyped--as in conventionalized, meaning that the action is done with some incidental features fixed; e.g. grinding spices could just as well have been grinding something else, but it's stereotyped to be grinding spices, and that secondary feature takes over the signaling role (so much that the spices aren't ev...
The Long Long Covid Post
by Zvi — 2022-02-10T13:10:01.452Z

A while back I mentioned I’d aim to write a longer post on Long Covid and Katja Grace’s post on it. This is that post. First I deal with Katja’s post, then Scott Alexander’s Long Covid: Much More Than You Wanted to Know.

My core model of Long Covid after writing this post:

Long Covid is real, but less common than many worry it is.
Reports of Long Covid are often people who have symptoms, then blame them on Long Covid whether or not they even had Covid. The exception is loss of taste and smell.
Long Covid severity and risk is proportional to Covid severity and risk.
If you didn’t notice you had Covid, you’re at very very low risk for developing Long Covid.
Vaccination is thus highly but incompletely protective against Long Covid.
Children are thus at minimal risk.
Omicron is thus less likely to cause serious Long Covid than Delta.
My current estimate of the forward-looking-practical-use chance of a healthy non-elderly person getting serious, life-impacting Long Covid from a case of Omicron is about 0.2%, or 1 in 500. This number will decline further once Paxlovid is readily available.
Long Covid remains the primary downside of contracting Covid while young and healthy.
Diseases often have long-term negative health effects. Long Covid is not fundamentally so different from Long Other Disease. If you are worried going forward about Long Covid you should consider things like permanently not living in a city to avoid diseases.
A lot of people are in poor health. It is likely worthwhile to treat your health a lot more seriously than most people do, irrespective of Covid.
The Precautionary Principle carries some weight in all this.
Remember that the chance of preventing a Covid case via additional Covid prevention, going forward, even with extreme measures, is not all that high.
If you compare the potential costs of Long Covid to the costs of Long Covid Prevention, it is obvious the second is a bigger threat.
Short-term additional vigilance is reasonable but rapidly becoming less reasonable.
Using Long Covid as a reason for not returning to normal once case levels come down would not be reasonable.

Katja’s Post

This may or may not be entirely fair, but I am going to use Katja Grace’s post on LessWrong as a steelman of the case for worrying about Long Covid. It is by a thinker I respect, and is clearly advocating for the side of ‘be worried,’ and seems to aim to be ex...
Your Enemies Can Use Your Prediction Markets Against You
by lsusr — 2022-02-10T10:37:01.037Z

It is the year 2525. Earth and Mars are in a state of cold war against each other. Both planets import uranium from Ceres. Ceres' government is pro-Earth…for now. Earth needs to know how likely it is the Ceres government will be overthrown and replaced by a pro-Mars regime.

Earth creates a prediction market on the blockchain. Earth seeds the prediction market by buying shares that pay out if there is a coup. This lopsided approach prevents Earth from directly subsidizing a coup it doesn't want.

Earth must provide lots of liquidity because if Earth provides too little liquidity then Mars can cheaply influence Earth's decisions by manipulating the prediction market's prices.

Ceres' allegiance is zero-sum. Mars wants there to be a coup because an anti-Earth regime is a pro-Mars regime and vice versa. Mars can subsidize a coup by buying shares that pay out if there is not a coup. Mars' investment simultaneously acts as an investment in a coup and insurance against the coup failing to materialize.

Suppose Mars values a coup at 5 million Dogecoin. Mars buys shares that pay out 5 million Dogecoin if there is not a coup. Mars' investment subsidizes a coup because entrepreneurial space pirates can become Mars' counterparty and use their expected profits to fund the conquest of Ceres. Even more efficiently, the Ceres government could buy the shares itself and then change its foreign policy to be pro-Mars, thereby realizing a risk-free profit without any violence at all (while also discouraging space pirates).

But from Mars' perspective it no longer matters if there is a coup because if there is a coup then Mars' windfall of 5 million Dogecoin exactly neutralizes the value it gains from a coup.

The less likely a coup is, the more Mars stands to gain by buying insurance. Suppose the prior implied probability of a regime change is 0.20. Mars can buy its shares for 1 million Dogecoin, pocketing a risk-free net utility equivalent to 4 million Dogecoin. If, alternatively, the prior implied probability of a regime change is 0.80 then Mars pockets a risk-free profit of only 1 million Dogecoin.

Where does this free money come from? Opportunity cost. The alternative to buying blockchain prediction market insurance is for Mars to send its own marines to overthrow the Ceres government. Mars landing troops on Ceres is only profitable if Mars does not buy insurance because insurance direc...
Inferring utility functions from locally non-transitive preferences
by Jan — 2022-02-10T10:33:18.433Z
Link: https://universalprior.substack.com/p/inferring-utility-functions

As part of the AI Safety Camp, I've been diving a bit deeper into the foundations of expected utility theory and preference learning. In this post, I am making explicit a connection between those two things that (I assume) many people already made implicitly. But I couldn't find a nice exposition of this argument so I wrote it up. Any feedback is of course highly welcome!

Preference utilitarianism and the Von-Neumann-Morgenstern theorem.

At the risk of sounding drab, I briefly want to revisit some well-trodden territory in the realm of expected utility theory to motivate the rest of this post.

When we are thinking about the question of how to align advanced AI with human values, we are confronted with the question of whether we want to capture "How humans act" or "How humans should act". Both approaches have virtue, but if we're being ambitious[1] we probably want to aim for the latter. However, our understanding of "How humans should act" is still rather confused and ready-made solutions to "plug into" an AGI are not available.

Rather than tackling the entire problem at once, we might focus on one particularly well-formalized portion of ethics first. A possible answer to the question "How should I act?" comes from

preference utilitarianism

where we focus on satisfying everyone's preferences (as revealed by their choices). I say that this portion of ethics is well-formalized because, it

turns out

, if you are being reasonable about your preferences, we can represent them succinctly in a "utility function," u(A)∈R. This utility function has the neat property that for two possible options, L and M, you prefer option M over option L iff u(M)>u(L).

This is the famous "

von-Neumann-Morgenstern theorem

" which sits at the heart of an approach of "doing good better".

Oskar Morgenstern and John von Neumann

Now given that the utility function lives in the realm of mathematics, there is a seemingly natural strategy to use it to steer AI. Get everyone's utility functions, combine them into a target function, and then let the AI pick the actions that increase the target function the most. If we have to pick a single number to maximize, there is a case to be made that utility is one of the best single numbers we can hope for.

Sounds good. Where is the catch?

The futility of computing utility.

Let's start by trying to wr...
EleutherAI's GPT-NeoX-20B release
by leogao — 2022-02-10T06:56:41.155Z
Link: http://eaidata.bmk.sh/data/GPT_NeoX_20B.pdf

GPT-NeoX-20B is a 20 billion parameter autoregressive language model whose weights will be made freely and openly available to the public through a permissive license. It is, to the best of our knowledge, the largest dense autoregressive model that has publicly available weights. In this paper, we describe the model architecture and training, evaluate its performance, and discuss the broader impacts of its release. We are open-sourcing the training and evaluation code, as well as the model weights, at https://github.com/EleutherAI/gpt-neox.

The alignment section of the paper would probably be especially of interest to this community.
The Defeat of Reason?
by M. Y. Zuo — 2022-02-10T04:29:07.864Z

Niels Bohr brainwashed a whole generation of theorists into thinking that the job of interpreting quantum theory was done 50 years ago.

Dr. Murray Gell-Mann

Re-reading Tim Maudlin’s famous book review at Boston Review on the fundamental problems of modern physics and of the enterprise of reason itself in relation to two noteworthy books gave me the opportunity to reflect on my own thoughts, half formed, on this subject which I have entertained for a while.

In the course of development of a technologically advanced society a well informed individual at the dawn of the 20 century would likely have believed that the scientific theories developed thus far and those to be developed would eventually advance likewise in sophistication, internal correctness, and correlation with observable reality, leading to ever greater predictive models. Which can then be applied to the practical arts in the form of engineering, political economics, social development, and human-societal uplifting. With the positive feedback loop reinforcing and accelerating the pace of development.

Certainly this was the expectation of many by the mid 20th century.

Now from the perspective of 2022 we can confidently analyze the events of the preceding century on Earth using retrospective and counter factual comparison and come to the conclusion that events did not develop entirely along a straightforward progression.

The reason why can be seen through the lens of the stagnation and profound contradictions of fundamental physics that has developed in the last few decades, or Maudlin so implies. Thus, we approach the first book:

The title of the first book is self explanatory: What Is Real?: The Unfinished Quest for the Meaning of Quantum Physics by Adam Becker

The theory reminds me a little of the system of delusions of an exceedingly intelligent paranoiac.

Albert Einstein on Niels Bohr’s theory (later becoming the Copenhagen Interpretation, which became commonly accepted as the default view on quantum physics )

Most notably in the field of physics, according to Maudlin, the stagnation is both profound and inexplicable according to the commonly accepted Copenhagen Interpretation, some may say even necessitating an anti-intellectual and anti-rational mindset to resolve the contradiction. This theory was developed by Niels Bohr and his followers in the first half of the 20th century, ...