trentmkelly/LessWrong-43k huggingface dataset:
AISafety.info "How can I help?" FAQ
by steven0461 — 2023-06-05T22:09:57.630Z

AI existential risk has been in the news recently. A lot of people have gotten interested in the problem and some want to know what they can do to help. Additionally, other existing routes to getting advice are getting overwhelmed, like

AI Safety Support

,

80,000 Hours

,

AGI Safety Fundamentals

,

AI Safety Quest

, etc. With this in mind, we’ve created a new FAQ as a part of

Stampy’s AI Safety Info

, based mostly on ideas from

plex

,

Linda Linsefors

, and

Severin Seehrich

. We're continuing to improve these articles and we welcome feedback.

By starting at the

root of the tree

and clicking on the articles at the bottom of each article, you can navigate to the article that most applies to your situation. It branches out into the rest of AISafety.info as well.

Or you can just look at the full list here:

I’m convinced that AI existential safety is important and want to contribute. What can I do to help?

I want to help out AI alignment without necessarily making major life changes. What are some simple things I can do to contribute?

I would like to focus on AI alignment, but it might be best to prioritize improving my life situation first. What should I do?

I want to take big steps to contribute to AI alignment (e.g. making it my career). What should I do?

How can I do conceptual, mathematical, or philosophical work on AI alignment?

How can I use a background in the social sciences to help with AI alignment?

How can I do organizational or operations work around AI alignment?

How can I work on AGI safety outreach in academia and among experts?

How can I work on public AI safety outreach?

How can I work on AI policy?

I’m interested in providing significant financial support to AI alignment. How should I go about this?

How can I work on assessing AI alignment projects and distributing grants?

How can I work on helping AI alignment researchers be more effective, e.g. as a coach?

What should I do with my idea for helping with AI alignment?

What subjects should I study at university to prepare myself for alignment research?

I’d like to do experimental work (i.e. ML, coding) for AI alignment. What should I do?

What should I do with my machine learning research idea for AI alignment?

Who should I talk to about my non-research AI alignment coding project idea?

How can I work toward AI alignment as a software engineer?

How can I do machine learning programmin

...
Answer to a question: what do I think about God's communication patterns?
by Jim Pivarski — 2023-06-05T21:40:19.665Z

This started as a comment on Book Review: How Minds Change, but it got so long and off the original topic that I felt it didn't belong in that comments feed. People would have to scroll past a large block of text that doesn't have much to do with the original article.

For context, I was responding to jaspax,

(Sorry, it doesn't look like the conservatives have caught on to this kind of approach yet.)

Actually, if you look at religious proselytization, you'll find that these techniques are all pretty well-known, albeit under different names and with different purposes. And while this isn't actually synonymous with political canvassing, it often has political spillover effects.

If you wanted, one could argue this the other way: left-oriented activism is more like proselytization than it is factual persuasion. And LessWrong, in particular, has a ton of quasi-religious elements, which means that its recruitment strategy necessarily looks a lot like evangelism.

when I said,

And even more deeply than door-to-door conversations, political and religious beliefs spread through long-term friend and romantic relationships, even unintentionally.

I can attest to this first-hand because I converted from atheism to Catholicism (25 years ago) by the unintended example of my girlfriend-then-wife, and then I saw the pattern repeat as a volunteer in RCIA, an education program for people who have decided to become Catholic (during the months before confirmation), and pre-Cana, another program for couples who plan to be married in the church (also months-long). The pattern in which a romantic relationship among different-religion (including no-religion) couples eventually ends up with one or the other converting is extremely common. I'd say that maybe 90% of the people in RCIA had a Catholic significant other, and maybe 40% of the couples in pre-Cana were mixed couples that became both-Catholic. What this vantage point didn't show me was the fraction in which the Catholic member of the couple converted away or maybe just got less involved and decided against being married Catholic (and therefore no pre-Cana). I assume that happens approximately as often. But it still shows that being friends or more than friends is an extremely strong motivator for changing one's views, whichever direction it goes.

Since it happened to me personally, the key thing in my case was that I didn't start

...
The Intrinsic Interplay of Human Values and Artificial Intelligence: Navigating the Optimization Challenge
by Joe Kwon — 2023-06-05T20:41:46.124Z

Looking for feedback; criticisms, requests for more evidence, expression of doubt, etc. all appreciated. I write about a bunch of the standard concepts from Alignment discourse, and don't expect most ideas here to be novel; I think an easily readable narrative/package for a broader audience could be useful though. Here I put an emphasis on the feedback loop between what humans care about and the optimizing systems interfacing with them. Thanks to Umarbek Nasimov for initial feedback.

Abstract: This post explores the complex interaction between human values and artificial intelligence (AI), focusing on the challenges and opportunities inherent in creating AI systems that can respect and adapt to human values. We begin by examining the formation of human values, rooted in evolutionary pressures and modulated by environmental influences. We then discuss how AI, as a powerful optimization tool, can shape human values, highlighting the potential risks associated with value-lock and the importance of preserving value diversity. We look into the implications of Goodhart's Law in the context of AI, illustrating the pitfalls of over-optimization and the difficulty in translating human values into robust optimization targets. It underscores the need for a multidisciplinary approach, continual adaptation to the dynamism of human values, and the importance of safeguards against over-optimization. Looking forward I emphasize the need for ongoing research, policy discussions, and technological advancements to navigate the future interplay of AI and human values. Through this exploration, the post encourages a thoughtful, proactive approach to AI development that respects the richness and complexity of human values.

Intrinsic Interplay Between Human Values and Environment

Formation of Values and Environmental Influence
The course of human evolution, under the influence of environmental pressures, has resulted in the encoding of specific adaptive inductive biases in our genetic makeup. These biases surface during early stages of cognitive development in infancy and lay the groundwork for the formation of values. These base-level tendencies are engaged through human experiences, which trigger the activation of genetically encoded reward circuits and provide positive neurochemical reinforcement for select behaviors. As individuals mature and expand their understanding...
The (local) unit of intelligence is FLOPs
by boazbarak — 2023-06-05T18:23:06.458Z

[Crossposting again on

Lesswrong

and

Windowsontheory

, with the hope I am not overstaying my welcome in LW.]

Wealth can be measured by dollars. This is not a perfect measurement: it’s hard to account for purchasing power and circumstances when comparing people across varying countries or time periods. However, within a particular place and time, one can measure wealth in the local currency. It still does not capture everything (e.g., future earnings, social connections). But generally, all else being roughly equal, the more dollars one has, the wealthier one is.

How do we measure intelligence? I am not interested in measuring the intelligence of individual humans or individual animals. Nor am I looking for a universal absolute scale of intelligence on which we could rank humans, elephants, and GPT4. (Indeed, it doesn’t seem that a one-dimensional comparison can be made; for example, we seem to be more intelligent than elephants on most dimensions, but they do have an

impressive memory

.) Rather, I want to compare different species within the same genus or different models within the same general architecture (e.g., Transformers).

I think it’s fair to say that the local unit of intelligence for animal species is neurons. While elephants have larger brains than humans, within the genus Homo, to a first approximation, the bigger the brain, the more intelligent the species.

(Figure from

Bolihus et al.

)

I claim that within the current architectures and training frameworks of large language models, the local unit of intelligence is FLOPs. That is, as long as we follow the current paradigm of training transformer-based architectures within best practices of scaling compute and data, the more compute resources (FLOPs) invested in training the model, the more intelligent it is. This is an imperfect measurement, but probably one that is better than trying to give models “IQ exams” that were designed for humans (and even there have

dubious value

). Another way to say this is that the intelligence of the model scales with the number of “load-bearing gradient steps” that have gone into training it.

So far, it might seem like a tautology, but as I claimed in the

“intelligence forklift” post

, this does have some implications. In particular, current general-purpose models such as ChatGPT are built in...
Tutor-GPT & Pedagogical Reasoning
by courtlandleer — 2023-06-05T17:53:09.753Z

This post hopes foster discussion about latent heutagogic potential in human+LLM collaboration and share some early results with the LessWrong community.

Spontaneous emergence of theory of mind capabilities in large language models hints at a host of overhung possibilities relevant to education. Metaprompting can elicit the foundation of such behavior--reasoning pedagogically about the learner.

Teachers constantly construct & revise robust psychological models of their students--good ones, subconsciously. For AI applications to deliver worthwhile learning experiences they'll need to excel in this regard.

TL;DR

We open-sourced "tutor-gpt," a digital Aristotelian learning companion.

What makes tutor-gpt compelling is its ability to posit from dialogue the most educationally-optimal tutoring response. Eliciting this from the capability overhang involves multiple chains of metaprompting enabling construction of a very nascent academic theory of mind for each student.

Most ‘chat-over-content’ tools in this recent explosion fail to capitalize on the enormous latent abilities of LLMs. Even the impressive out-of-the-box capabilities of contemporary models don’t achieve the necessary user intimacy. Not-dangerous infrastructure for that doesn’t exist yet.

And the advent of personal, agentic AI for all is going to need it. So to that end, we see releasing tutor-gpt's architecture into the wild as one small step on the journey to supercharge the kind of empowering generative agents we want to see in the world.

Neo-Aristotelian Tutoring

Right now, tutor-gpt is a reading comprehension and writing workshop tutor. You can chat with it in Discord or spin-up your own instance. After supplying it a passage, it will coach you toward understanding or revising a piece of text. It does this by treating the user as an equal, prompting and challenging socratically.

We started with reading and writing in natural language because (1) native language acumen is the symbolic system through which all other fluencies are learned, (2) critical dialogue is the ideal vehicle by which to do this, and (3) that's what LLMs are best at right now.

The problem is, most students today don't have the luxury of "talking it out" with an expert interlocutor. But we know that's what works.

(Perhaps too) heavily referenced in tech and academia, Bloom’s 2 sigma problem suggests that students tutored 1:1 can perfor...
Not another bias!
by Lionel — 2023-06-05T17:50:05.981Z
Link: https://lionelpage.substack.com/p/not-another-bias

Behavioural economics has significantly advanced our understanding of human behaviour, shedding light on how it frequently diverges from the traditional economic model predictions of the homo economicus. There is however a growing sense, that the marginal returns of finding and investigating biases have decreased significantly.

There is a renewed interest in adaptive explanations for biases and cognitive biases. This interest is bolstered by progress in at least two different areas: cognitive neuroscience and economic theory. In the former, models assuming that human brains are solving complex Bayesian calculations are now being routinely used. In the latter, richer models studying optimal/equilibrium behaviour in situations where information is incomplete and costly to acquire have enriched economists' ability to study more realistic decision problems with the economic toolbox.

I recently published a book, "Optimally Irrational", that discusses the results from behavioural economics with an adaptive perspective that takes from old and recent explanations of human biases as having good reasons. I also started an eponym Substack where I'll be discussing human behaviour with an adaptive perspective, informed by recent advances in economic theory and cognitive neuroscience. I think these would be of interest to the LessWrong community.
What I've been reading, June 2023
by jasoncrawford — 2023-06-05T17:08:41.805Z
Link: https://rootsofprogress.org/reading-2023-06

A monthly feature. As usual, recent blog posts and news stories are omitted from this; you can find them in my links digests. In all quotes below, any emphasis in bold was added by me.

Books

Thomas S. Ashton, The Industrial Revolution, 1760–1830 (1948). A classic in the field, concise and readable. Crafts (see paper below) cites this work as pointing out “the links between scientific thought and the industrial revolution” that were later synthesized by Mokyr. Given that I’ve already read a lot on this topic, there were no big revelations here, but many interesting details, such as the effect of interest rates on infrastructure building. War tended to raise interest rates and thus to slow growth.

Samuel Butler, Erewhon (1872). I picked this one up because it has an early prediction of the machines taking over, was cited by authors including Turing and Haldane, and presumably inspired Dune’s “Butlerian jihad.” I expected a dystopian sci-fi novel, but it’s actually a political satire, and quite amusing, although I need some exegesis to understand exactly what he is satirizing. I’m only halfway through, though, and haven’t gotten to the “war on the machines” yet.

James Pethokoukis, The Conservative Futurist: How to Create the Sci-Fi World We Were Promised (forthcoming). Pethokoukis is very well-read in the progress field, and I love his quotations and sources; reading his stuff always sends me off on a bunch of interesting followup reading. It was from his blog, for instance, that I discovered American Genesis. Several of the books mentioned below were quoted or cited here.

H. G. Wells, Anticipations Of the Reaction of Mechanical and Scientific Progress upon Human Life and Thought (1901). I think both Pethokoukis and J. Storrs Hall referenced this one. I’ve only lightly sampled it, but it has some very interesting predictions about the future of transportation and other technologies. E.g., this explanation for why motor vehicles are needed:

Railway travelling is at best a compromise. The quite conceivable ideal of locomotive convenience, so far as travellers are concerned, is surely a highly mobile conveyance capable of travelling easily and swiftly to any desired point, traversing, at a reasonably controlled pace, the ordinary roads and streets, and having access for higher rates of speed and long-distance travelling to specialized ways restricted to swift traffic, a

...
Humans don't understand how we do most things
by Nathan1123 — 2023-06-05T14:35:43.543Z

Hello,

I find that a fundamental premise towards the development of AI is the ability to dissect all the internal calculations used by the human mind to make rational decisions, and then replicate those calculations in an electronic system. On a surface level, this works very well for many kinds of problem solving. In our everyday lives, we use logical reasoning to make rational decisions all the time, with the understanding that some decisions come from emotion and intuition instead. There are also some cognitive behavior that is completely subconscious or autonomous (such as walking or talking), which ends up being the most challenging skills to replicate for a machine.

For any kind of occupation a human performs (whether it is a doctor, writer, soldier, engineer, etc.), the skills they use at that job come in one of two forms: transferable skills and non-transferable skills. A transferable skill is effectively a skill where the rational calculations and algorithms needed to perform it are fully understood and documented, and therefore can be easily instructed to someone with a sufficient amount of intelligence. A non-transferable skill is a skill whose rational calculations and algorithms are not fully understood. Humans are still capable of performing that skill (expertly so), but this ability comes from sheer experience rather than any instructions.

Because we live in a universe that is rationally-consistent, then it must be concluded that all skills must have some algorithm behind it, regardless if it is transferable or not. Therefore, human experience must somehow construct the algorithms necessary to perform a non-transferable skill, but these calculations are done subconsciously, and thus appears from an outside perspective to be sheer intuition.

In designing artificial intelligence, a transferable skill is the easiest to program: as long as we understand the algorithms used by our own mind, then it is trivial to design that same algorithm in software. A computer is essentially a blank slate that will absorb whatever instructions is given to it.

The problem is that any occupation that is worth paying money for must have some non-transferable skills. If a job had nothing but transferable skills, then their role is easily replaceable by anyone who can be taught to do the same thing (including a machine). So a fundamental reason that career jobs exist at al...
Wildfire of strategicness
by TsviBT — 2023-06-05T13:59:17.316Z

It may not be feasible to make a mind that makes achievable many difficult goals in diverse domains, without the mind also itself having large and increasing effects on the world. That is, it may not be feasible to make a system that strongly possibilizes without strongly actualizing.

But suppose that this is feasible, and there is a mind M that strongly possibilizes without strongly actualizing. What happens if some mental elements of M start to act strategically, selecting, across any available domain, actions predicted to push the long-term future toward some specific outcome?

The growth of M is like a forest or prairie that accumulates dry grass and trees over time. At some point a spark ignites a wildfire that consumes all the accumulated matter.

The spark of strategicness, if such a thing is possible, recruits the surrounding mental elements. Those surrounding mental elements, by hypothesis, make goals achievable. That means the wildfire can recruit these surrounding elements toward the wildfire's ultimate ends. By recruiting more dry matter to the wildfire, the wildfire burns hotter and spreads further.

Also by hypothesis, the surrounding mental elements don't themselves push strongly for goals. Seemingly, that implies that they do not resist the wildfire, since resisting would constitute a strong push. We can at least say that, if the totality of the mental elements surrounding the wildfire is going to notice and suppress the wildfire, it would have to think at least strategically enough to notice and close off all the sneaky ways by which the wildfire might wax. This implies that the surrounding mental elements do a lot of thinking and have a lot of understanding relevant to strategic takeovers, which itself seemingly makes more available the knowledge needed for strategic takeovers.

Capabilities overhang provides fuel for a wildfire of strategicness. That implies that it's not so easy to avoid wrapper-minds.

This is very far from being a watertight argument, and it would be nice if the conclusion were false; how is it false? Maybe, as is sometimes suggested, minds selected to be creative are selected to keep themselves open to new ideas and therefore to new agency, implying that they're selected to prevent strategic takeovers, which would abhor new agency?
Speaking off-meta
by Epirito — 2023-06-05T13:56:26.107Z

I'm sure you have been in this situation:
You want to say something, but it doesn't sound right. So you reword it in a way that

gets the point across,
you feel better about saying and
is strangely worded.

So you say it.
This is very common.
But why say it so oddly?
Is it because you wanted to be really precise? Yes, sometimes, but mostly not, I think.
Are you really struggling to say exactly what you mean, or are you just struggling to bypass your intuition that you shouldn't say it at all?
Some illustrations:

ChatGPT jailbreak

Someone asks ChatGPT to do something. "As a large language model, I cannot do so and so."
Then they reword it or try to gaslight it into thinking that it can.
And it turns out that it can sort of do it after all. It's just that the developers didn't want it to happen.

Foreign lyrics

Do foreign lyrics sound less cringey or are your cringe detection systems underdeveloped for the foreign language in question because of your lack of fluency?

Walter White

There are many tense situations in Breaking Bad. Sometimes, the protagonist, Walter White, sees his life is at risk. Whenever feels anxious or threatened, he wants to solve everything by talking and talking. As he himself says:

but she just won't... she just won't understand. I mean, no matter how well I explain it, these days she just has this... this... I mean, I truly believe there exists some combination of words. There must exist certain words in a certain specific order that can explain all of this, but with her I just can't ever seem to find them.

In conclusion, I want to say that speaking off-meta is often counterproductive.
You might have managed to feel good about saying it right now, but your knee-jerk responses are sometimes wiser than you can see while deliberating what to say right now. You have only an instant to decide what to say, but the other person has the rest of their life to see the evil or disease in your soul that caused you to say what you said (as long as they can remember that you said it). So, in the long-term, you don't even save face, you just fool yourself.

However I'm still not sure if there is some real use for it that I'm overlooking. Maybe there are some situations when it's enough to sound ok at the moment? I don't know. What do you think?