trentmkelly/LessWrong-43k huggingface dataset:
ChatGPT and Bing Chat can't play Botticelli
by Asha Saavoss — 2023-03-29T17:39:50.029Z

Botticelli is a guessing game similar to Twenty Questions. It's a simple game that I sometimes play with my 7 year-old daughter. I tried to get ChatGPT and Bing Chat to play Botticelli with me, but neither tool could do it.

How to play Botticelli

Description of the game

Botticelli is a lot like Twenty Questions, where a "chooser" secretly picks a person and a guesser asks Yes/No questions to figure out who the chooser's person is. At the start of the game, the chooser reveals the letter that their chosen person's last name starts with.

Here is the big difference between Botticelli and Twenty Questions. Before the guesser gets to ask a Yes/No question about the chooser's secret person, the guesser must first stump the chooser. The guesser tries to stump the chooser by thinking of their own person whose name also starts with the specified letter, and then asking the chooser a question for which this person is a valid answer.

The chooser is NOT stumped if they come up with a person who is both (1) a valid answer to the guesser's question; and (2) whose last name starts with the same letter as the chooser’s person. The person the chooser comes up with does not have to be the same person the guesser had in mind.

However, if the chooser cannot think of a valid answer to the guesser's question, then the chooser was stumped. In that case, the guesser may ask one Yes/No question to learn about the chooser's person.

This process is repeated until the guesser has enough information to figure out the chooser's person.

Example game

Chooser: I have chosen a person. Their name starts with a “C.”

Guesser: I will try to stump you. Is your person a former U.S. president?

Chooser: No, my person is not Jimmy Carter.

Guesser: I was thinking of Bill Clinton, but your answer works too. I didn’t stump you, so I will try again. Is your person a famous actor?

Chooser: Hmmm. I can’t think of anyone.

Guesser: I was thinking of Tom Cruise.

Chooser: I see. You stumped me. Now, you can ask a yes/no question about my person.

Guesser: Okay. Here’s a yes/no question about your person. Is your person a musician?

Chooser: Yes. My person is a musician.

Guesser: Okay. I will try to stump you again. Did your person sing the song “Hit the Road Jack?”

Chooser: You got it! My person was Ray Charles.

Guesser: Yay. I win.

ChatGPT and Bi

...
The Rationalist Guide to Hinduism
by Harsha G. — 2023-03-29T17:03:41.107Z
Link: https://somestrangeloops.substack.com/p/the-rational-guide-to-hinduism

The Lost schools of Thought in Hinduism and What They Mean.

TL;DR: Hinduism, the world's 3rd/4th largest belief system, has surprising, rational roots that go beyond idolatry and polytheism. This piece unravels the richness of its intellectual heritage and first principles thinking, dissecting various schools of thought – from the 6 classical to 4 non-traditional branches (covering monotheism, polytheism, pantheism, atheism, nihilism, and hedonism and a bunch of other isms). Also learn why these schools collapsed and the ”growth hacking” techniques that led to the idolatory and prayer to win over other schools.

Hinduism (or Sanatana Dharma if you're feeling fancy) is a bit of a chameleon — it can take on a bunch of different shapes and sizes depending on who's practicing it.

It's a buffet of beliefs, with options ranging from hedonism (all about pleasure) to dharma (duty and morals over everything else), from meditation to prayer, from theism (god exists) to atheism (god doesn’t exist).

It forms a strange loop of interconnected ideas and practices, referencing one another, forming a unique world of faith that's hard to define. But we'll give it a shot anyway

Part I. The Truth of Truth of Hinduism

Hinduism's got a bunch of philosophical systems and schools.

There are 6 major traditional schools with 4 major sects following ideas from these schools. And there are 4 other non-traditional schools.

The Traditional schools in Part 1 follow the Vedas and Upanishads and are called Astika. We’ll cover 6 of these below. We've got 6 big schools and 4 major sects (some would say cults) using ideas from these schools.

Then there are the "Non-Traditional" schools, or Nastika, including Charvakas, Ajivikas, Buddhism, and Jainism, mentioned in Part 3.

These ideas popped up (or at least got written down) around 1500-500 BCE, after the Indus Valley Civilisation went bust. But they're probably older than that.

Heads up: Some "Hindus" think Nastika means atheism. This isn’t true. Even Astika schools can be atheistic. Nastika just means they don’t believe in Vedas to be their core texts. Some Nastika schools (Charvaka) are atheistic, while others (Buddhism, Jainism) traditionally don’t comment on the whole god thing too much.

1.1. The 6 Major Schools (And Why You Only Know One)

Hinduism's got 6 classic systems, the "Shad Darshanas."

They talk about metaphys...
“Unintentional AI safety research”: Why not systematically mine AI technical research for safety purposes?
by Jemal Young — 2023-03-29T15:56:39.355Z

Thanks to llll for helping me think this through, and for providing useful comments.

Epistemic Status: My best guess

Introduction

It might be worthwhile to systematically mine AI technical research to find “unintentional AI safety research”—research that, while not explicitly conducted as AI safety research, contains information relevant to AI safety. An example of unintended safety research is Douglas Lenat's work on the heuristic-search system

Eurisko

, which inadvertently demonstrated specification gaming when Eurisko exploited a loophole in the rules of the role-playing game Traveller TCS to win the US national championship in 1981 and 1982.[1] This post is not meant to suggest that AI safety researchers don’t already look for unintentional safety research, but I’m unaware of any effort to do so in a systematic way designed to extract as much “safety value” as possible from technical research.

Related work

Tshitoyan, Vahe, et al. "Unsupervised word embeddings capture latent knowledge from materials science literature." Nature 571.7763 (2019): 95-98. In this study, a language model trained on an unsupervised text corpus of scientific abstracts learned to predict discoveries that were made after the training data cutoff date—that is, the model learned to predict future scientific discoveries. “This suggests that latent knowledge regarding future discoveries is to a large extent embedded in past publications. Our findings highlight the possibility of extracting knowledge and relationships from the massive body of scientific literature in a collective manner, and point towards a generalized approach to the mining of scientific literature.”

Reasons to systematically mine technical research

Gleaning insights from technical research has already demonstrated its value for safety researchers, even without a structured approach. For example, technical researchers demonstrated in practice that reward misspecification, bias in training data, distributional shift, and adversarial examples can cause ML models to behave in surprising or unintended ways,[2] and demonstrations like these have illuminated safety research directions. A less straightforward example probably worth mentioning is the discovery of “unit-level” feature visualization in DNNs[3]—a proof of concept for mechanistic interpretability, which is an area of general purpose research where...
Strong Cheap Signals
by trevor — 2023-03-29T14:18:52.734Z
Link: https://betonit.substack.com/p/strong-cheap-signals

By Bryan Caplan, Economics professor at George Mason University, posted on his blog Bet on It on March 16th 2022. This is the original source of the legendary quote.

A basic tenet of game theory is that strong favorable signals must be costly. Why? Because if strong favorable signals weren’t costly, everyone would send them. And if everyone sends strong signals, they cease to be strong.

Recently, however, I realized that this tenet isn’t as airtight as I thought.

The problem, in a nutshell, is that most people are impulsive. As a result, they’re prone to casually send bad signals – and even more prone to casually fail to send good signals.

Example: Having friends is extremely helpful in life. As I often chant, “Friendship rules the world.” To make new friends, however, you want to signal your friendliness. The obvious strategic implication: Everyone should aggressively signal their friendliness to potentially useful people with warm smiles, amiable greetings, and kind hospitality.

Since friends are scarce and wonderful, you’d expect this signaling race to be intense. But a half century of experience tells me otherwise. Most people are rather standoffish – especially young people, who have the greatest need to make friends! Even if you’re obviously useful for them, people rarely strive to win you over. Indeed, most don’t even bother to give a smile and an audible greeting.

Would friendliness actually be a good use of people’s time? Almost certainly. Research says that people get half of all jobs through personal connections. And you’re highly unlikely to do a favor for a person who doesn’t even know your name.

How can this situation persist? Because most human beings are both shy and prideful. They fear rejection. And they don’t like to do anything that resembles “sucking up.” The wise consciously bring these impulses under control – and soon start to make friends and influence people. Fortunately for everyone else, the wise are few in number. As a result, plenty of opportunities for the shy and prideful remain. But they would do far better if they just got their impulses under control.

At this point, you could fairly ask, “Wait! Does this mean that seemingly friendly people are actually just people with high impulse control?” No. While most human beings are shy and prideful, a notable minority are sincerely friendly. The pool of people we perceive as fri

...
Missing forecasting tools: from catalogs to a new kind of prediction market
by MichaelLatowicki — 2023-03-29T09:55:30.885Z

Introduction

It’s possible to make quantitative forecasting (forecasting by software) significantly more easy and fun by adding some missing pieces to the forecasting ecosystem. Let’s do it.

The models I’m interested in are

“gears” and not “behavior”

models: They should be motivated by theory, be built to inform our theoretical understanding, and have relatively few numeric parameters. The point is to do science, not just fit parameters to data.

In this post I focus on forecasting things that are regularly measured and published by institutions. Such data includes economic and population statistics, opinion and value surveys, statistics about violence, public health, etc. I propose that we can build tools that would make it easier to test theories through forecasting models.

For example, a simple model for national meat consumption would use data about GDP growth per capita, population growth, population value judgements (as measured by the World Value Survey) and the past year's meat consumption to predict next year's consumption. Similar inputs would be relevant for predicting carbon emissions. Predicting GDP would presumably rely on corruption indicators, past growth, educational attainment etc.

Motivation

make it easier to build, share and use models

There is no “Wikipedia for predictive models” that I know of. No big repository to easily share and find predictive scientific models other than the relevant domain’s scientific literature, which is not optimized for these tasks: it is not organized by the variables being predicted, it is not generally available as reusable and modular software components, it is usually not focused on predictive work, some of it is paywalled, etc.

low-effort empiricism for non-experts

Evaluating the empirical performance of predictive models is a noisy, but low-bias, way to access the truth. People who aren't domain experts attempt to access the truth by finding people they hope are knowledgeable and then trusting their opinion, because doing research yourself is hard. Think about the people you disagree with regarding the safety of vaccines. Consider how bad these people are in identifying reliable domain experts, and how dangerous that weakness is. We can create tools even low-information users can utilise to get in touch with reality.

The tools we miss

There are several tools we can build to make quantitative forecasting ...
Spreadsheet for 200 Concrete Problems In Interpretability
by Jay Bailey — 2023-03-29T06:51:46.114Z

Thanks to the Open Source Mechanistic Interpretability Slack for their feedback.
When going through Neel's excellent 200 Concrete Problems In Interpretability sequence, I found myself thinking "It sure would be great to have a single document I could go through to see all the problems, and ideally sort them by difficulty as well."

So I made a spreadsheet! Problems are numbered by category, a feature which should be added to the main sequence fairly soon as well.
The primary purpose of the spreadsheet is to let people quickly scan through to find problems that they're interested in before diving more deeply. The secondary purpose of it is to let people avoid duplication of work / find collaborators, so if you know of work that's already been done on some of these problems, or you're actively working on one and want help, please leave a comment on the sheet so I can add it in!
Which parts of the existing internet are already likely to be in (GPT-5/other soon-to-be-trained LLMs)'s training corpus?
by AnnaSalamon — 2023-03-29T05:17:28.000Z

In particular:

If a book is in easily accessible parts of the web, but is there as a pdf image of a scanned book rather than as text, is it likely to be in the training corpus of (GPT5/similar) in roughly the same way (with roughly the same effects) as if it were in easily accessible webpages?
How about if a book is on libgen as a pdf (illegally), but not on the easily accessible parts of the web?
How about if the ideas are in a video (e.g. a podcast); is (GPT5/similar) likely to process the text content of a video in roughly the same way if there's only a video, compared to if there's a video and a transcript?
Are there specific books that it might slightly help alignment to have on the internet?
by AnnaSalamon — 2023-03-29T05:08:28.364Z

Books, and ideas, have occasionally changed specific human beings, and thereby history. (I think.)

I used to think it utterly implausible when people suggested that "AIs are our kids, we need to raise them right" or that e.g. having the right book written about (ethics/philosophy/decision theory/who knows) might directly impact an AI's worldview (after the AI reads it, in natural language) and thereby the future. But, while I still consider this fairly unlikely, it seems not-impossible to me today. Future LLMs could AFAICT have personalities/belief-like-things/temporary-unstable-values-like-things/etc. that're shaped by what's on the internet. And the LLMs' initial personalities/beliefs/values may then change the way they change themselves, or the way that social networks that include the LLMs help change the LLMs, if and when some LLMs self-modify toward more power.

So I have "what books or ideas might help?" in my shower-thoughts.

One could respond to this possibility by trying to write the right ethical treatises or train-of-thought interface or similar. More cheaply, one could respond to this by asking if there are books that've already been written that might be at least a little bit helpful, and whether those books are already freely available online and within the likely training corpuses of near-future LLMs, and if not, whether we can easily cause them to be.

Any thoughts on this? I'll stick my own in the comments. I'll be focusing mostly on "what existing books might it help to cause to be accessibly online, and are there cheap ways to get those books to be accessibly online?", but thoughts on other aspects of these questions are also most welcome.
FLI open letter: Pause giant AI experiments
by Zach Stein-Perlman — 2023-03-29T04:04:23.333Z
Link: https://futureoflife.org/open-letter/pause-giant-ai-experiments/

We call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4.

AI systems with human-competitive intelligence can pose profound risks to society and humanity, as shown by extensive research and acknowledged by top AI labs. As stated in the widely-endorsed Asilomar AI Principles, Advanced AI could represent a profound change in the history of life on Earth, and should be planned for and managed with commensurate care and resources. Unfortunately, this level of planning and management is not happening, even though recent months have seen AI labs locked in an out-of-control race to develop and deploy ever more powerful digital minds that no one – not even their creators – can understand, predict, or reliably control.

Contemporary AI systems are now becoming human-competitive at general tasks, and we must ask ourselves: Should we let machines flood our information channels with propaganda and untruth? Should we automate away all the jobs, including the fulfilling ones? Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us? Should we risk loss of control of our civilization? Such decisions must not be delegated to unelected tech leaders. Powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable. This confidence must be well justified and increase with the magnitude of a system's potential effects. OpenAI's recent statement regarding artificial general intelligence, states that "At some point, it may be important to get independent review before starting to train future systems, and for the most advanced efforts to agree to limit the rate of growth of compute used for creating new models." We agree. That point is now.

Therefore, we call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4. This pause should be public and verifiable, and include all key actors. If such a pause cannot be enacted quickly, governments should step in and institute a moratorium.

AI labs and independent experts should use this pause to jointly develop and implement a set of shared safety protocols for advanced AI design and development that are rigorously audited and overseen by independent outside experts. These protocol

...
Run Posts By Orgs
by jefftk — 2023-03-29T02:40:01.586Z

I'm very happy to see effective altruism community members write
public posts about EA organizations, where they point out errors,
discuss questionable choices, and ask hard questions. I'd like to see
more cases where someone digs deeply into an org's work and writes
about what they find; a simple "this checked out" or a "critical
feedback that helps the org improve" are both good outcomes. Even a
"this org is completely incompetent" is a good outcome: I'd rather
orgs be competent of course, but in the cases where they aren't we
want to know so we can explore alternatives.

When posting critical things publicly, however, unless it's very
time-sensitive we should generally be letting orgs review a draft first. This
allows the org to prepare a response if they want, which they can post
right when your posts goes out, usually as a comment. It's very
common that there are important additional details that you don't have
as someone outside the org, and it's good for people to be able to
review those details alongside your post. If you don't give the org a
heads up they need to choose between:

Scrambling to respond as soon as possible, including working on
weekends or after hours and potentially dropping other commitments,
or

Accepting that with a late reply many people will see your post,
some will downgrade their view of the org, and most will never see the
follow-up.

Surprising them is to your advantage, if you consider this tactically,
but within the community we're working towards the same goals:
you're not trying to win a fight, you're trying to help us all get
closer to the truth.

In general I think a week is a good amount of time to give for
review. I often say something like "I was planning to publish this on
Tuesday, but let me know if you'd like another day or two to review?"
If a key person is out I think it's polite to wait a bit longer (and
this likely gets you a more knowledgeable response) but if the org keeps
trying to push the date out you've done your part and it's fine to say
no.

Sometimes orgs will respond with requests for changes, or try to engage
you in private back-and forth. While you're welcome to make edits in
response to what you learn from them, you don't have an obligation to:
it's fine to just say "I'm planning to publish this as-is, and I'd be
happy to discuss your concerns publicly in the commen...
Desensitizing Deepfakes
by worse — 2023-03-29T01:20:41.540Z

There is some discussion on the forum about using AI to detect whether or not something is a deepfake, and perhaps some trust in anti-deepfake bots to be better resourced etc. in this arms race. But could we give ourselves a bit of breathing room here?

Could it be incredibly valuable to accelerate desensitization to deepfakes? Or at least make people more aware of them by using humor?

It seems like a real risk that someone eventually creates a convincing and harmful deepfake, a la current President saying, "America has launched nuclear weapons against Russia." Or vice versa or literally anything bad, with voice and video, that to our eyes is Very Real and Convincingly Terrible.

Should we be subverting people's expectations by familiarizing them with deepfakes, perhaps best done by example? If you have not seen it already, it seems that the memes of the US Presidents playing computer games (Warning expletives+: US Presidents play Minecraft) is actually a pretty good example of this. On the flip side of this, I also recall seeing a video being shown to an older individual who, despite Biden saying the most heinous things, found it more believable that it was real than a deepfake.

So maybe spamming content for significant figures doing whacky things is effective for updating people's models for the probability of a deepfake. I was considering further pushing the bounds here by walking the fine line of a 'real fake' nuclear assault, a la Biden saying, "My fellow Americans, we have launched nukes against... the Moon." But that seems unnecessarily too close to the mark.

There could be an info hazard that by demonstrating the capabilities of deepfakes you actually show people that this is a powerful manipulation tool and increase risk of malicious use.

Unsure call to action here, thought I'd share, hope for steelmanning/meta-feedback on post, and just indicate a potentially impactful angle that seems to be working pretty well in some ways...

Desensitizing idea of deepfakes without risk, in a way that may make people more likely to question whether something said is real (could be worth more rigorous study).
Large language models aren't trained enough
by sanxiyn — 2023-03-29T00:56:13.925Z
Link: https://finbarr.ca/llms-not-trained-enough/

Finbarr Timbers makes a point, obvious in retrospect, but which many people, including people forecasting AI timeline, seem to miss: since training cost is amortized over inference, optimal training depends on expected amount of inference. Both scaling laws from OpenAI and DeepMind assume zero (or negligible) inference, which is obviously incorrect. Any forecasting using scaling laws similarly is suspect and should be revised.