trentmkelly/LessWrong-43k huggingface dataset:
The Opposite Of Autism
by Capybasilisk — 2022-03-27T15:30:24.921Z

It's probably not presuming too much to guess that many around here have personal experience with the autism spectrum, if not in relation to themselves, then with close family. I say this because the kinds of subjects discussed around here are exactly the type that would appeal to those of an autistic persuasion, e.g. technical systems, logic, and (arguably) utilitarian philosophy. Many here probably have backgrounds in STEM, and those fields tend to have a significant over-representation of people on the spectrum.

An issue that often comes up in software design (a field with high ASD representation) is programmers not being able to properly model the wants and needs of non-technical end-users. I bring this up because I see AI alignment as being a scaled-up version of this problem. The kind of people who have a strong interest in AI/machine learning will likely have a greatly disproportional impact on the future of human civilization. This might be a problem as not only is this subset of humans highly atypical in cognitive style, but the very mental architecture which underlies their interest in technical systems restricts their ability to model the minds of typical humans!

The hardest humans for ASD types to model would be those with minds that are the diametric opposite of their own. Call this condition anti-autism. It would consist of...well I'm not exactly sure. It's hard for me to imagine the mental lives of these people. I've heard the phrase "people vs things" thrown around, implying that ASD types are drawn to inanimate objects, and humans who are on the opposite side of this condition would be drawn to people. I'm not so sure. I think that plenty of people with ASD have an obsessive interest in categorizing humans and other living things.

While there's been a great amount of study behind autism, there's a curious lack of interest in what a condition with its exact opposite traits looks like, or even the fact of its existence. Simon Baron-Cohen, one the most famous autism researchers and creator of the systemizer-emphathizer scale [0], argued that it doesn't come up because it's not actually a problem. Basically, a human who is system-blind but extremely skilled at reading other humans (hyper-empathy, in his terms) can get by perfectly well [1]:

Scientists have never got up close to these individuals. It is a bit like positing the existence of a new an

...
Practical everyday human strategizing
by  — 2022-03-27T14:20:19.389Z

Reply to: Humans are not automatically strategic

AnnaSalamon writes:

[Why] do many people go through long training programs “to make money” without spending a few hours doing salary comparisons ahead of time? Why do many who type for hours a day remain two-finger typists, without bothering with a typing tutor program? Why do people spend their Saturdays “enjoying themselves” without bothering to track which of their habitual leisure activities are *actually* enjoyable? Why do even unusually numerate people fear illness, car accidents, and bogeymen, and take safety measures, but not bother to look up statistics on the relative risks?

I wanted to give a practical approach to avoiding these errors. So, I came up with the following two lists. To use these properly, write down your answers; written language is a better idea encoding tool than short-term memory, in my experience, and being able to compare your notes to your actual thinking sometime later is useful.

Goal Accuracy Checking

What do you think is your current goal?
How would you feel once you achieve this goal?
Why is this useful to you and others, and how does it relate to your strengths?
Why will you fail completely at achieving this goal?
What can you do not to fail as badly?

Reasoning behind these question choices:

If you have no aim, you are by definition aimless. By asking for current goals rather than long-term goals, we can specify context-specific actions and choices which would allow better execution. For instance, my goal could be “to be a better writer” or “to think more rationally”. More specific and time-limited goals seem to work better here, specifically if they can be broken down into sub-goals.
An analogous choice here would be pleasure or reward maximization in the future. Generally speaking, it does not seem to be the case that we are motivated by wanting to feel worse in the future. So if your goal allows you to feel better about yourself, or to improve the material conditions of your immediate community—that seems to be a good indicator that it is worthwhile. I would like to hear what people think about this, though, as some goals might show their self-destructive tendencies based on the feelings they induce when they are finished. An example: creating your own business and working 18 hours every day might feel “good” insofar as it allows you to feel productive a

...
Manhattan project for aligned AI
by Chris van Merwijk — 2022-03-27T11:41:00.818Z

One possible thing that I imagine might happen, conditional on an existential catastrophe not occurring, is a Manhattan project for aligned AGI. I don’t want to argue that this is particularly likely or desirable. The point of this post is to sketch the scenario, and briefly discuss some implications for what is needed from current research.

Imagine the following scenario: It is only late that top AI scientists take the existential risk of AGI seriously, and there hasn't yet been a significant change in the effort put into AI safety relative to our current trajectory. At some point, there is a recognition among AI scientists and relevant decision-makers that AGI will be developed soon by one AI lab or another (within a few months/years), and that without explicit effort there is a large probability of catastrophic results. A project is started to develop AGI:

It has an XX B$ or XXX B$ budget.
Dozens of the top AI scientists are part of the project, and many more assistants. People you might recognize or know from top papers and AI labs join the project.
A fairly constrained set of concepts, theories and tools are available that give a broad roadmap for building aligned AGI.
There is a consensus understanding among management and the research team that without this project, AGI will plausibly be developed relatively soon, and that without explicitly understanding how to build the system safely it will pose an existential risk.

It seems to me that it is useful to backchain from this scenario to see what is needed, assuming that this kind of alignment Manhattan project is indeed what should happen.

Firstly, my view is that if this Manhattan project would start in intellectual conditions similar to today’s, there wouldn't be very many top AI scientists significantly motivated to work on the problem, and it would not be taken seriously. Even very large sums of money would not suffice, since there wouldn't be enough of a common understanding about what the problem is for it to work.

Secondly, it seems to me that there isn't enough of a roadmap for building aligned AGI for such a project to succeed in a short time-frame of months to years. I expect some people to disagree with this, but looking at current rates of progress in our understanding of AI safety, and my model of the practical parallelizability of conceptual progress, I am skeptical that the problem c...
[ASoT] Some ways ELK could still be solvable in practice
by leogao — 2022-03-27T01:15:16.607Z

Editor’s note: I’m experimenting with having a lower quality threshold for just posting things even while I’m still confused and unconfident about my conclusions, but with this disclaimer at the top.

This post is a followup to my earlier post.

If ELK is impossible in generality, how could we solve it in practice? Two main ways I can think of:

Natural abstractions

: maybe the complexity of direct translators is bounded in practice because the model ontology is not so different from ours, violating the worst case assumption.

Science being easy

: maybe there isn’t any hard ceiling to how good you can get at doing science, and it's not actually that much more expensive so it is actually feasible to just scale science up.

For the first one, I feel like there are both reasons for and against it being potentially true. The main argument for, especially in the context of LMs, is that human abstractions are baked into language and so there's a good chance LMs also learn this abstraction. I

used to be

a lot more optimistic that this is true, but since then I've changed my mind on this. For one, in the limit, natural abstractions are definitely not optimal. The optimal thing in the limit is to simulate all the air molecules or something like that. So we know that even if natural abstractions holds true at some point, it must eventually stop being optimal, and the abstractions get more and more weird past that point. Then the question becomes not just whether or not there even exists a sweet spot where natural abstractions holds, but also whether it overlaps with the kinds of models that are dangerous and might kill us. I still think this is plausible but kind of iffy.

I think this is worth

looking into empirically

, but I'm very cautious about not reading too much into the results, because even if we see increasingly natural abstractions as size increases, it could still start going down again before our models are sufficiently powerful. I'm moderately worried that someone will overclaim the significance of results in this vein. Also, my intuition about language models is that despite communicating in language, they learn and think in pretty alien ways.

For the second one, if this is true then this kind of defeats most of the point of even doing ELK. If you can just scale up the science doing, then ELK becomes just an optimization the same way that reward modelli...
What are the top 1-10 posts / sequences / articles / etc. that you've found most useful for yourself for becoming "less wrong"?
by Aryeh Englander — 2022-03-27T00:37:45.180Z

Posts can be from LessWrong, the EA Forum, or any other blog or forum. Articles from other sources are fine as well, as are videos or podcast episodes. Books don't count for the purpose of this post, but if there is a specific chapter in a book that you found particularly useful then that counts.
Beyond Blame Minimization
by physicaleconomics — 2022-03-27T00:03:31.650Z

Like a lot of people, I've become interested in the performance of public health bureaucracies over the course of the COVID-19 pandemic. But while I've seen a lot of discussion of the specific flaws and failures of the FDA, the CDC, the WHO, etc., I haven't seen much discussion of the logic of bureaucracy in general.

What I'm hoping to find, or at least to better understand why I can't find it, is a general theory of bureaucracy, in the same way that economics already has for people, firms and democratic politicians. Any time we want to explain what a person, a firm, or a democratic politician does, we say they are a utility-maximizer, a profit-maximizer, and a median-voter-grabber, respectively. These models aren't necessarily perfect, but they are a clear starting point for almost any analysis of the respective subjects. They offer a broad, general understanding consistent across many contexts and time periods.

The power of these broad, general models is that they give a very clear starting point for analyzing problems and solutions to those problems. When individuals are choosing badly, change the incentives. When businesses are choosing badly, tax or subsidize something—change what's profitable. When politicians are choosing badly, change what voters want. These solutions aren't necessarily easy to design or implement. But it's nice as a base and a starting point.

Having access to these kinds of very general explanatory models minimizes the kind of confusion that many of us may have felt watching the decisions of public health authorities—a sense of "Bwuh?!" When a person talks about giving back to the community but never donates to charity, I made not be happy with them, but I can understand it in terms of utility maximization. When a firm pollutes the air or makes a product with cheap, unreliable parts, I may not be happy with them, but I can understand it in terms of profit maximization. When a politician adopts an extreme position in the primaries and then moves to a moderate position during the general election, I may not be happy with them, but I can understand it in terms of the median voter model. But the choices of bureaucracy are confusing. Even after being consistently surprised by them, it's relatively hard to not be surprised by the next thing they do.

Without access to these models, I think that a sense of "Bwuh?!"...
A Primer on God, Liberalism and the End of History
by Mahdi Complex — 2022-03-26T20:07:58.074Z

Medieval Christendom

Medieval Europe’s political life was extremely fractured. The old continent was ruled by a variety of often feuding kings, emperors, lords and princes. Yet, political and cultural life still had a certain level of cohesiveness thanks to unifying force of Christianity and the Catholic Church. Indeed, the Church provided medieval Europe with a common rule of law by setting, for example, rules for marriage and the inheritance of property. This allowed the Church to eventually gain possession of between a quarter and third of all lands in most European countries.

In the late eleventh century, the church also gained the power to appoint bishops, after what would become known as the investiture controversy, a power struggle lasting a few decades between popes and Holy Roman Emperors. Thus, the Church managed to establish itself as an important power broker, commanding loyalties and exercising sovereignty in a manner that transcended borders and traditional allegiances.

The investiture controversy is illustrative of Christianity’s awkward relationship with politics, and the tensions that have existed between secular rulers and religious authorities, ever since Christianity was made the official religion of the Roman Empire in 380 CE. Due to its origins as a minority sect under Roman rule, Christianity offers little guidance in the crafting of a just Christian political order. Unlike in Islam, there is no example for the Christian ruler to follow.

And yet, unexpectedly thrust into the position of dominant religion in Europe, religious and secular rulers had to wrestle with the question of how to apply Christian doctrines to political life. After all, how could a religion purporting to have knowledge of the will of the creator of the Cosmos – the Lord of the Heavens and the Earth – not have anything authoritative to say on the matter of politics?

Thus, Christian political-theology has been characterized by the tension between the “City of Man,” the temporal realm of nature, suffering, sin, and political necessities, and what Saint Augustine called the “City of God,” the eternal realm of salvation, grace and divine justice.

Luther

Europe’s precarious religious unity was shattered with the publication of Martin Luther’s Ninety-five Theses in 1517. He challenged the authority of the pope, and a number of doctrines of an increasingly corrupt Catholic Ch...
Agency and Coherence
by David Udell — 2022-03-26T19:25:21.649Z

Epistemic status: spitballing.

"Like Photons in a Laser Lasing"

When you do lots of reasoning about arithmetic correctly, without making a misstep, that long chain of thoughts with many different pieces diverging and ultimately converging, ends up making some statement that is... still true and still about numbers! Wow! How do so many different thoughts add up to having this property? Wouldn't they wander off and end up being about tribal politics instead, like on the Internet?

And one way you could look at this, is that even though all these thoughts are taking place in a bounded mind, they are shadows of a higher unbounded structure which is the model identified by the Peano axioms; all the things being said are true about the numbers. Even though somebody who was missing the point would at once object that the human contained no mechanism to evaluate each of their statements against all of the numbers, so obviously no human could ever contain a mechanism like that, so obviously you can't explain their success by saying that each of their statements was true about the same topic of the numbers, because what could possibly implement that mechanism which (in the person's narrow imagination) is The One Way to implement that structure, which humans don't have?

But though mathematical reasoning can sometimes go astray, when it works at all, it works because, in fact, even bounded creatures can sometimes manage to obey local relations that in turn add up to a global coherence where all the pieces of reasoning point in the same direction, like photons in a laser lasing, even though there's no internal mechanism that enforces the global coherence at every point.

To the extent that the outer optimizer trains you out of paying five apples on Monday for something that you trade for two oranges on Tuesday and then trading two oranges for four apples, the outer optimizer is training all the little pieces of yourself to be locally coherent in a way that can be seen as an imperfect bounded shadow of a higher unbounded structure, and then the system is powerful though imperfect because of how the power is present in the coherence and the overlap of the pieces, because of how the higher perfect structure is being imperfectly shadowed. In this case the higher structure I'm talking about is Utility, and doing homework with coherence theorems leads you to appreciate that we only k

...
Magic, tricks, and high-dimensional configuration spaces
by pchvykov — 2022-03-26T19:14:00.176Z

What is a magic trick? Any complex system, such as our world, may be described in terms of its numerous degrees of freedom, each designated along its own configuration dimension. As such, the configuration of such a system may be captured by a long list of numbers – or, equivalently, by a point in a high-dimensional configuration space. In this post, I want to entertain the idea that magic tricks are just exploiting these “extra-dimensions” to do the “impossible” - leading to some broad implications!

How can extra dimensions help to do magic? Think of shadow theater: a stick can cast a line-shadow on my screen – which looks like a wall to my little shadow character. If I move my character past that stick, on the screen it will appear that they moved “through the wall.” So something that looks impossible in 2D space, was easily accomplished with the help of an extra dimension. Just so, when a magician apparently “teleports a coin from a matchbox to behind my ear,” they actually do something quite simple, but using extra dimensions - ones that I did not notice were there, like their sleeve. The art of the magician, in this case, is to manipulate my attention in such a way that out of all the myriads of the degrees of freedom describing our complex world, I only pay attention to a few, and in particular that I miss the ones actually being used to do the trick.

This simple idea seems quite universal to me – and perhaps can fruitfully explain many things we consider wonderous in the world, like technology, genius, or "real" magic. Computers work because of the hidden electronic degrees of freedom that have been carefully engineered for the task. A brilliant diplomat might find a collaborative dimension between two enemies that no one else noticed. In all cases, the game is about “finding a way to walk around the wall, when others may have tried to walk through it.” I.e., for a task that seems hard or impossible according to the familiar dimensions of the world, try to find a new dimension that might make the task easy.

As such, it seems to me that what we might call “real" magic is just another example of this, where the extra dimension being utilized is entirely new and unfamiliar to us – something we have never seen or experienced before, and thus something we can never guess at. Thus, a computer will be truly magical to someone who has never hea...
Compute Governance: The Role of Commodity Hardware
by Jan — 2022-03-26T10:08:07.518Z
Link: https://universalprior.substack.com/p/compute-governance-the-role-of-commodity?s=w

TL;DR: Thoughts on whether CPUs can make a comeback and become carriers of the next wave of machine learning progress (spoiler: they probably won't). Meditations on challenges in compute governance.

Over the coming decades,

we might see

machines that are as smart or smarter than humans. It is possible that these smarter-than-human machines will be built to work on our behalf and to greatly increase human welfare, such as by curing diseases and expanding economic opportunity. But simultaneously there are high risks if we build these machines with goals that are too narrow and that don't represent our broader values. The field concerned with working towards positively shaping artificial intelligence (AI) is called AI Safety (

Wiblin, 2017

).

There are two complementary avenues for ensuring that humanity successfully navigates the transition into a world with advanced AI systems:

Technical AI Safety

and

AI Governance

. While Technical AI Safety is concerned with developing algorithms and frameworks for making safe AI possible, AI governance is concerned with how institutions make decisions about AI and with ensuring that AI is deployed safely (

Dafoe 2020

).

Compute Governance is the subfield of AI Governance concerned with controlling and governing access to computational resources (

Heim 2021

;

Anderljung and Carlier 2021

). Among different candidates for effective AI governance, Compute Governance stands out as particularly promising due to two factors:

We can monitor possession and usage of large-scale computing resources comparatively well. Large-scale computing requires physical space for the computing hardware and substantial supporting infrastructure.
Access to large-scale computing resources can be monitored and governed as the supply chain for advanced semiconductor production is concentrated on a few key producers.

In this post, I will explore possible implications on Compute Governance of a recently proposed technique for accelerating progress in AI research (

Chen et al. 2019

).

State of Compute Governance, “AI-specialized” and “commodity” hardware

To properly contextualize the proposed technique, I will briefly summarize the state of Compute Governance. A series of reports from the Center for Security and Emerging Technology (

CSET

) examines the supply chain for computing resources (

Khan et al.,2021

). In particular, the reports

highlight governable hardware cho

...