trentmkelly/LessWrong-43k huggingface dataset:
Feminism and Femininity
by  — 2022-05-18T22:53:18.174Z

Is it feminist to wear lipstick? There are four possible views on the position that feminist thought should take with respect to expressions of conventional femininity in patriarchal societies:

Expressing femininity is actively good
Expressing femininity is actively bad
Women should do whatever makes them feel most comfortable
Femininity is a distraction - masculinity is the problem

These different views express a combination of different views on the mechanism by which patriarchy operates, the most effective way to advance women's liberation, and the duties which women have to their gender.

How does patriarchy work

There are three competing views of how patriarchy might work. Patriarchy could be a result of gender socialisation. This theory goes that the way in which gender is socialised holds women back from holding power, achieving status and having good life chances - or some combination of all three. In contemporary Western Europe this mostly manifests as relatively soft norms where, for instance, women are more harshly punished for expressing anger. Probably the most important substantiation of gender socialisation as preventing women from achieving power and status - although not necessarily good life outcomes - is the norm where women sacrifice their careers when raising children with a male partner. In less gender equal societies this can take on a much more extreme form, where the common pattern is the barring of women from the public sphere. This describes societies such as classical Athens or 19th century Pashtun farmers where women are excluded from commerce and politics - the things that give men power.

The competing view is that patriarchy is defined by a value system rather than a set of norms. In the former theory what was deemed to be high status things - the axis along which inequality should be measured - was taken as a given. It was participating in the public square. Under this other view however, patriarchy comes from the work women do and femininity per se being defined as less valuable than the work men do and masculinity per se.

There’s a strong version and a weak version of this claim. The strong version says that it really would be fine if women stayed within the private sphere if that work was valued equally to the work in the public sphere. To make this concrete, and staying within the context of capitalist ...
What's up with the recent monkeypox cases?
by johnswentworth — 2022-05-18T22:06:24.370Z

Is it a thing I should be allocating attention to?

Wikipedia tells me that human-to-human transmission of monkeypox is pretty rare/difficult. There has been some community spread recently at least in the UK, but that's been speculated to have been from sex. On the other hand, there's been a bunch of cases reported in Portugal and Spain recently as well. Is that just normal background rate being given more attention than usual by the news cycle? Or is this actually an unusually high number of cases? Most importantly, is the number of cases significant evidence of increased human-to-human transmission?
We have achieved Noob Gains in AI
by phdead — 2022-05-18T20:56:49.143Z

TL;DR I explain why I think AI research has been slowing down, not speeding up, in the past few years.

How have your expectations for the future of AI research changed in the past three years?
Based on recent posts in this forum, it seems that results in text generation, protein folding, image synthesis, and other fields have accomplished feats beyond what was thought possible. From a bird's eye view, it seems as though the breakneck pace of AI research is already accelerating exponentially, which would make the safe bet on AI timelines quite short.

This way of thinking misses the reality on the front lines of AI research. Innovation is stalling beyond just throwing more computation at the problem, and the forces that made scaling computation cheaper or more effective are slowing. The past three years of AI results have been dominated by wealthy companies throwing very large models at novel problems. While this expands the economic impact of AI, it does not accelerate AI development.

To figure out whether AI development is actually accelerating, we need to answer a few key questions:

What has changed in AI in the past three years?

Why has it changed, and what factors have allowed that change?

How have those underlying factors changed in the past three years?

By answering these fundamental questions, we can get a better understanding of how we should expect AI research to develop over the near future. And maybe along the way, you'll learn something about lifting weights too. We shall see.

What has changed in AI research in the past three years?

Gigantic models have achieved spectacular results on a large variety of tasks.

How large is the variety of tasks? In terms of domain area, quite varied. Advances have been made in major hard science problems like protein synthesis, imaginative tasks like creating images from descriptions, and playing complex games like Starcraft.

How large is the variety of models used? While each model features many domain specific model components and training components, the core of each of these models is a giant transformer trained with a variant of gradient descent, usually ADAM.

How large are these models? That depends. DALLE2 and AlphaFold are O(10GB), AlphaStar is O(1GB), and the current state of the art few shot NLP models (Chinchilla) are O(100GB).

One of the most consistent findings of the past decade of AI research is that lar...
A possible check against motivated reasoning using elicit.org
by david reinstein — 2022-05-18T20:52:35.601Z

Are you worried you may be engaging in motivated reasoning, rationalization ... or committing other reasoning fallacies?
I propose the following epistemic check using Elicit.org's "reason from one claim to another" tool
Whenever you have a theory that A→B,
Take your theory negating one side or the other (or the contrapositive of either negation), and feed it into this tool.
Feed it
A→¬B
and/or
¬A→B,
and see if any of the arguments it presents seem equally plausible to your arguments for A→B.
If they seem similarly plausive, believe your original arguments and conclusion less.
Caveat: the tool is not working great yet, and often requires a few rounds of iteration, selecting the better arguments and tell.ing it "show me more like this", or feeding it some arguments.

When in Rome ... do or don't do as the Romans do?
Reading the ethicists: A review of articles on AI in the journal Science and Engineering Ethics
by Charlie Steiner — 2022-05-18T20:52:20.942Z

Epistemic status: Stream of consciousness reactions to papers read in chronological order. Caveat lector.

I have a dirty family secret. My uncle is a professional ethicist.

In a not-too roundabout way, this is why I ended up looking at the October 2020 issue of the journal Science and Engineering Ethics, their special issue on the ethics of AI. I am now going to read that issue, plus every article this journal has published about AI since then [I wussed out and am just going to skim the latter for ones of special interest] and give you the deets.

October 2020

Hildt et al., Editorial: Shaping Ethical Futures in Brain-Based and Artificial Intelligence Research

This is the introduction to the issue. They give each paper a sentence or two of summary and try to tie them all together. The authors helpfully give a list of topics they think are important:

• Data Concerns: Data management, data security, protection of personal data, surveillance, privacy, and informed consent.

• Algorithmic Bias and Discrimination: How to avoid bias and bias related problems? This points to questions of justice, equitable access to resources, and digital divide.

• Autonomy: When and how is AI autonomous, what are the characteristics of autonomous AI? How to develop rules for autonomous vehicles?

• Responsibility: Who is in control? Who is responsible or accountable for decisions made by AI?

• Questions relating to AI capabilities: Can AI ever be conscious or sentient? What would conscious or sentient AI imply?

• Values and morality: How to build in values and moral decision-making to AI? Are moral machines possible? Should robots be granted moral status or rights?

Based on this list, I anticipate that I'm about to run into four-sixths ethics papers about present-day topics that I will skim to point out particularly insightful or anti-insightful ones, one-sixth philosophers of mind that I will make fun of a little, and one-sixth papers on "How to build values into general AI" that I'm really curious as to the quality of.

Onward!

Nallur, Landscape of Machine Implemented Ethics

Primarily this paper is a review of a bunch of papers that have implemented or proposed ethics modules in AI systems (present-day things like expert systems to give medical advice, or lethal autonomous weapons [which he has surprisingly few qualms about]). These were mostly different varieties of rule-following or constraint-sa...
Maxent and Abstractions: Current Best Arguments
by johnswentworth — 2022-05-18T19:54:44.255Z

This post is not-very-distilled and doesn’t contain much background; it’s intended for people who already have the context of

at least

these

four

posts

. I’m putting it up mainly as a reference for people who might want to work directly on the math of natural abstractions, and as a technical reference post.

There’s various hints that, in most real-world cases, the distribution of low-level state given high-level natural abstractions should take the form of a maximum entropy distribution, in which:

The “features” are sums over local terms, and
The high-level variables are (isomorphic to) the Lagrange multipliers

More formally: we have a low-level causal model (aka Bayes net) P[XL]=∏iP[XLi|XLpa(i)]. Given the high-level variables XH, the distribution of low-level variable values should look like

P[XL|XH]=1ZP[XL]eλT(XH)∑ifi(XLi,XLpa(i))

… i.e. the maximum-entropy distribution subject to constraints of the form E[∑ifi(XLi,XLpa(i))|XH]=μ(XH). (Note: λ, fi, and μ are all vector-valued.)

This is the sort of form we see in statistical mechanics. It’s also the form which the

generalized Koopman-Pitman-Darmois (gKPD) theorem

seems to hint at.

I don’t yet have a fully-satisfying general argument that this is the main form which abstractions should take, but I have two partial arguments. This post will go over both of them.

Maxent Telephone Argument

Two different nested layers of Markov blankets on the same underlying causal DAG

Quick recap of the Telephone Theorem: information about some variable X passes through a nested sequence of Markov blankets M1,M2,…. Information about X can only be lost as it propagates. In the limit, all information is either perfectly conserved or completely lost. Mathematically, in the limit P[X|Mn]=P[X|Fn(Mn)] for some F such that Fn(Mn)=Fn+1(Mn+1) with probability approaching 1 as n→∞; F is the perfectly-conserved-in-the-limit information carrier.

In this setup, we can also argue that the limiting distribution limn→∞P[X|Mn] should have a maxent form. (Note: this is a hand-wavy argument, not a proper proof.)

Think about how the distribution (x↦P[X=x|Mn]) transforms as we increment n by 1. We have

P[X|Mn+1]=∑MnP[X|Mn]P[Mn|Mn+1]

First key property of this transformation: it’s a convex combination for ...
Framing Practicum: Dynamic Programming
by aysajan — 2022-05-18T18:16:38.496Z

This is a framing practicum post. We'll talk about what dynamic programming (DP) is, how to recognize DP in the wild, and what questions to ask when you find it. Then, we'll have a challenge to apply the idea.

Today's challenge: come up with 3 examples of DP which do not resemble any you've seen before. They don't need to be good, they don't need to be useful, they just need to be novel (to you).

Expected time: ~15-30 minutes at most, including the Bonus Exercise.

What is DP?

Suppose I am about to drive from Miami to Boston and I need to get to Boston as fast as possible. As a first step, I check the highway map and create a list of possible routes for this trip (let’s assume “good” old times with no Google maps). For instance, looking at the imaginary routes in the figure below, the route at the top says I should take the “Miami → A → C → E → G → Boston” route and the total trip distance would be 200+170+220+400+430=1420 miles. Which specific route should I take to minimize the total distance, thus total travel time? I can, of course, calculate total travel distance for each possible route and pick the one with least-distance. But it could easily get very time consuming if there exist hundreds of thousands of possible routes to evaluate.

One alternative approach to identify a route with minimum distance is to use a backward method. Suppose I drive backward through the route map from right to left. First, I will start at the destination, Boston. If I am in city G or H, I have no further decision to make since there is only one route that leads me to Boston from either city. The number in green summarizes total distance with one last trip, or one stage, to go.

My first decision (from right to left) occurs with two trips, or stages to go. If, for example, I am in city F, I can either drive 150 miles to city G and another 430 miles to Boston - total 580 miles, or drive 200 miles to city H and another 400 miles to Boston - total 600 miles. Therefore, the shortest possible distance, or optimal route (in green), from city F is 580 miles (F → G → Boston). The alternative route from F (F → H → Boston) is suboptimal with 600 miles to go (in red).

Let me back up one more city, or stage, and compute the least-distance from city C and D to Boston. Figure below summarizes these calculations.

Once I have computed the optimal route from city C and D onward to Bosto...
How to get into AI safety research
by Stuart_Armstrong — 2022-05-18T18:05:06.526Z

Recently, I had a conversation with someone from a math background, asking how they could get into AI safety research. Based on my own path from mathematics to AI alignment, I recommended the following sources. It may prove useful to others contemplating a similar change in career:

Superintelligence by Nick Bostrom. It condenses all the main arguments for the power and the risk of AI, and gives a framework in which to think of the challenges and possibilities.

Sutton and Barto's Book: Reinforcement Learning: An Introduction. This gives the very basics of what ML researchers actually do all day, and is important for understanding more advanced concepts. It gives (most of) the vocabulary to understand what ML and AI papers are talking about.

Gödel without too many tears. This is how I managed to really grok logic and the completeness/incompleteness theorems. Important for understanding many of MIRI's and LessWrong's approaches to AI and decision theory.

Safely Interruptible agents. It feels bad to recommend one of my own papers, but I think this is an excellent example of bouncing between ML concepts and alignment concepts to make some traditional systems interruptible (so that we can shut them down without them resisting the shutdown).

Alignment for Advanced Machine Learning Systems. Helps give an overall perspective on different alignment methods, and some understanding of MIRI's view on the subject (for a deeper understanding, I recommend diving into MIRI's or Eliezer's publications/writings).

You mileage may vary, but these are the sources that I would recommend. And I encourage you to post any sources you'd recommend, in the comments.
A bridge to Dath Ilan?  Improved governance on the critical path to AI alignment.
by Jackson Wagner — 2022-05-18T15:51:10.293Z

Summary: This post showcases my 2nd-place-winning entry in the Future of Life Institute's AI worldbuilding contest. It imagines:

How we might make big improvements to decisionmaking via mechanisms like futarchy and liquid democracy, enhanced by Elicit-like research/analysis tools.
How changes could spread to many countries via competition to achieve faster growth than rivals, and via snowball effects of reform.
How the resulting, more "adequate" civilization could recognize the threat posed by alignment and coordinate to solve the problem.

(Cross-posted to the EA Forum)

Part of a mural illustrating our scenario, created by Diana Gurvich!

Motivation for our scenario:

Human civilization's current ability to coordinate on goals, make wise decisions quickly, and capably execute big projects, seems inadequate to handle the challenge of safely developing aligned AI. Evidence for this statement can be found practically all around you, but the global reaction to covid-19 is especially clarifying. Quoting Gwern:

The coronavirus was x-risk on easy mode: a risk (global influenza pandemic) warned of for many decades in advance, in highly specific detail, by respected & high-status people like Bill Gates, which was easy to understand with well-known historical precedents, fitting into standard human conceptions of risk, which could be planned & prepared for effectively at small expense, and whose absolute progress human by human could be recorded in real-time happening rather slowly over almost half a year while highly effective yet cheap countermeasures like travel bans & contact-tracing & hand-made masks could—and in some places did!—halt it. Yet, most of the world failed badly this test; and many entities like the CDC or FDA in the USA perversely exacerbated it, interpreted it through an identity politics lenses in willful denial of reality, obstructed responses to preserve their fief or eek out trivial economic benefits, prioritized maintaining the status quo & respectability, lied to the public “don’t worry, it can’t happen! go back to sleep” when there was still time to do something, and so on. If the worst-case AI x-risk happened, it would be hard for every reason that corona was easy. When we speak of “fast takeoffs”, I increasingly think we should clarify that apparently, a “fast takeoff” in terms of human coordination means any tak

...
Understanding Gato's Supervised Reinforcement Learning
by lorepieri — 2022-05-18T11:08:10.885Z
Link: https://lorenzopieri.com/rl_transformers/

The recent publication of Gato spurred a lot of discussion on wheter we may be witnessingth the first example of AGI. Regardless of this debate, Gato's makes use of recent developments in reinforcement learning, that is using supervised learning on reinforcement learning trajectories by exploiting the ability of transformer architectures to proficiently handle sequential data.

Reading the comments it seems that this point created some confusion to readers not familiar with these techniques. Some time ago I wrote an introductory article to how transformers can be used in reinforcement learning which may be helpful to clarify some of these doubts: https://lorenzopieri.com/rl_transformers/
Gato's Generalisation: Predictions and Experiments I'd Like to See
by Oliver Sourbut — 2022-05-18T07:15:51.488Z

I'm deliberately inhabiting a devil's advocate mindset because that perspective seems to be missing from the conversations I've witnessed. My actual fully-reflective median takeaway might differ.

My covid has made writing difficult at the moment, and I haven't had the energy to gather citations or fully explain the detail for some of the assertions in this post.

I believe (for various reasons not detailed here) that qualitative advances in general agentic performance from artificial systems are likely this decade and next - I just don't think Gato represents progress in that direction. I'm not particularly surprised by anything in the Gato paper[1]. Naturally, then, I'm against hyperbole around the capabilities demonstrated.

There is not enough information in the paper to say either way, but it may be the case that Gato represents a novel and cutting-edge distillation technique and if so, I would prefer if it were framed that way than as an increment towards generalisation ability of artificial agents!

The authors do a reasonable job of reporting their experiments (though I sense some motivated reasoning in the presentation of some results[2]), but the PR version of the story (at least as I have encountered it) departs from the mostly-honest paper itself[3].

I focus here almost entirely on sequential control, since that is the aspect of the discussion around the Gato paper which seems most clouded, while being at the centre of the purported generalisation, and in which I have the most interest and expertise.

Techniques and phenomena

We've known for many years that attention mechanisms (early 2010s) allow parameter sharing over sequences and over inputs of variable size/shape, that behavioural cloning (early 1990s?) often works with enough expert examples, that representation transfer (embeddings, early 2010s?) works, and that fine-tuning parameters (2000s? 2010s?) is faster than learning from scratch.

What should the key empirical takeaway be?

In my reading, if you subtract the already-known phenomena, and pass over the mostly-ignored distillation aspect, there is an important empirical finding in the Gato paper. I'm not sure if it's novel per se, but it probably is at this scale in this domain.

Namely, in our current systems, fine-tuning is typically fast when pretraining includes closely related tasks (e.g. DM Control Suite and Meta-World), slow when pretraining...
I just watched the Open C3 Subcommittee Hearing on Unidentified Aerial Phenomena (UFOs). Here's a succinct summary and commentary + some background
by mako yass — 2022-05-18T04:15:11.681Z

Background on UFOs, Disclosure, and Rationality

There have been reports of various strange flying objects for a very very long time (read the paragraph with "Boeotia". Note, a "hogshead" seems to be a type of barrel.), but, for me, it wasn't until quite recently that it became really unambiguously clear to me that something is really going on there. I'd recommend looking into the Princeton/Nimitz "tic-tac" incidents specifically. IIRC, at least 6 navy staff on those boats have very publicly and extensively testified to having seen a very strange propulsion technology. I've come across no story as to how, or why any human faction would be keeping a propulsion technology like that secret, and out of deployment for so long.
(A half-baked theory though: Perhaps this propulsion tech could be used to make devilishly fast, non-interceptible ICBMs, the existence of which would make the world worse, so maybe it's a technology that we should avoid publicly acknowledging for as long as possible?)

However, it's possible that it wasn't a propulsion technology, and that it's more of a plasma image projector used in combination with a radar spoofing technology, as futurist David Brin seems to confidently believe. In that case it's a lot easier to understand how the technology could have been kept hidden - it hasn't been mature for very long, and it has fairly limited applications, and it isn't salacious enough to leak about.
So this would be one of my dominant hypotheses, the tic-tacs really look like they're just this, to me. But there are a few contrary details in the account of commander Fravor and US officials (including Obama, among other former presidents) swear up and down that they'd tell us if that were it.

So I kinda have to keep paying attention to this stuff.

But isn't it wildly implausible? How can a rationalist entertain the possibility that aliens would be like this, or behave like this? The econ/tech-eschatology of it doesn't make any sense!

I'm sympathetic to that. My intuition says that if aliens had reached us, their construction projects would have filled our night sky, they wouldn't be hiding from us, because whatever the value of hiding, the costs are greater. Robin Hanson talks extensively about the dynamics of economically plausible aliens and finds that it wouldn't generally support these sorts of observations.

But the observations seem to be disagreeing w...