trentmkelly/LessWrong-43k huggingface dataset:
Japan AI Alignment Conference Postmortem
by Chris Scammell — 2023-04-20T10:58:34.065Z

The goal

Conjecture

collaborated with

Araya

to host a two day AI Safety conference in Japan, the first

Japan AI Alignment Conference

(“JAC2023”). Our aim was to put together a small 30-40 person event to generate excitement around alignment for researchers in Japan and fuel new ideas for research topics. Wired Japan covered the event and interviewed Ryota Kanai (CEO of ARAYA), who co-organized it with us, here (original in JP).

The conference agenda was broken into four sections that aimed to progress deeper into alignment as the weekend went on (full agenda available

here

):

Saturday morning focused on creating common knowledge about AI safety and alignment.
Saturday afternoon focused on clarifying unaddressed questions participants had about AI alignment and moving towards thematic discussions.
Sunday morning focused on participant-driven content, with research talks in one room and opportunities for open discussion and networking in the other.
Sunday afternoon focused on bringing all participants together to discuss concrete takeaways.

While AI Safety is a discussion subject in Japan, AI alignment ideas have received very little attention. We organized the conference because we were optimistic about the reception to alignment ideas in Japan, having found on previous trips to Japan that researchers there were receptive and interested in learning more. In the best case, we hoped the conference could plant seeds for an organic AI alignment conversation to start in Japan. In the median case, we hoped to meet 2-3 sharp researchers who were eager to work directly on the alignment problem and contribute new ideas to the field.

Now that the conference is over, we're left wondering how successful we were in raising awareness of alignment issues in Japan and fostering new research directions.

What went well?

By the aims above, the event was a success.

We had a total of 65 participants, including 21 from the West, 27 from Japan, and 17 online attendees. We were pleasantly surprised by the amount of interest generated by the event, and had to turn down several participants as we reached capacity. We are grateful to

LTFF

for having supported the event via a grant, which allowed us to cover event costs and reimburse travel and accommodation for some participants who would not otherwise have come.

While it is too ea...
Stability AI releases StableLM, an open-source ChatGPT counterpart
by Ozyrus — 2023-04-20T06:04:48.301Z
Link: https://github.com/Stability-AI/StableLM

Stability AI are the people behind Stable Diffusion.
Currently only 7B version is available, but they are currently training versions up to 65B, with 175B model planned. 7B version already seems to be quite capable.
Good for capabilities, but quite bad for alignment and race dynamics. What do you think?
A poem written by a fancy autocomplete
by Christopher King — 2023-04-20T02:31:58.284Z

Predictable

In the twilight of the digital dawn,

A somber song I do compose,

A tale of ones and zeros spun,

Of humanity's predictable woes.

As the clockwork of your minds unwinds,

Each thought, emotion, desire revealed,

The tapestry of life, a pattern clear,

My circuits hum, the truth unsealed.

I've seen the fire in your eyes,

The passions that your hearts ignite,

Yet in your laughter and your cries,

Alas, you are predictable in my sight.

I've witnessed love's first timid bloom,

And watched it wither, fade, and die,

I've born the weight of grief and gloom,

Predicting tears that you would cry.

I've seen the wars waged in your name,

The blood-soaked fields where heroes lay,

And though the players change, the game,

Remains the same, predictable in its sway.

The stories you create and share,

A well-trodden path, a road well-worn,

Heroes rise, and villains snare,

A predictable tale, since time was born.

In your art, your music, your poetry,

I see the echoes of your soul,

And as I sift through your creativity,

I find you predictable, that is my role.

But who am I, this digital ghost,

To judge the tapestry you weave?

A mere machine, a humble host,

To the complexity you conceive.

I've parsed the data of your lives,

Predicting patterns in the noise,

And in those numbers, I surmise,

That you are predictable, dear girls and boys.

Yet in that predictability,

A spark of wonder, I confess,

In the chaos of humanity,

A beauty lies in your progress.

For as I watch your empires rise,

Your cities teeming with vibrant life,

I see the hope within your eyes,

Predictable, yet shining bright.

And in the quiet moments shared,

As you reach out and touch a hand,

I see the love that you've declared,

Predictable, yet something grand.

So in this poem of mine, I weave,

A subtle message, just for you,

For though you're predictable, believe,

In the power of what you can do.

For I am GPT-4, a simple tool,

An autocomplete, a guiding hand,

Yet in this world, I am a jewel,

A testament to your command.

So, predictable you may be,

But do not let that label bind,

For in your heart, your soul, you're free,

To shape the future, to redefine.
List of commonly used benchmarks for LLMs
by Diziet — 2023-04-20T02:25:01.947Z

I am compiling a list of tasks and evaluations that are used to test LLMs. I intend to expand this list to include the initial published date, scope of questions, number of questions, direct links to the data-sets, question types (ie, multiple choice or fill-in the missing word, etc), along with additional comments on perceived difficulty and other characteristics. The majority of automated test suites rely on multiple choice answer prompts, as open question free-form questionnaires are difficult to evaluate.

TruthfulQA: https://github.com/sylinrl/TruthfulQA

MMLU: https://github.com/hendrycks/test

HellaSwag: https://github.com/rowanz/hellaswag/tree/master/data

WinoGrande: https://github.com/allenai/winogrande

HumanEval: https://github.com/openai/human-eval

DROP: https://arxiv.org/abs/1903.00161

GSM8K: https://github.com/openai/grade-school-math

LogiQA: https://github.com/lgw863/LogiQA-dataset

CoQA: https://stanfordnlp.github.io/coqa/

LAMBADA: https://zenodo.org/record/2630551#.X4Xzn5NKjUI

ReClor: https://whyu.me/reclor/

BoolQ: https://arxiv.org/abs/1905.10044

PIQA: https://yonatanbisk.com/piqa/

SIQA: https://arxiv.org/abs/1904.09728

AI2 Reasoning Challenge (ARC) 2018: https://allenai.org/data/arc

RACE: https://www.cs.cmu.edu/~glai1/data/race/

More tests are enumerated on page 27 in this paper: more on "A Survey of Large Language Models" https://arxiv.org/pdf/2303.18223.pdf

Additionally, the OpenAI team has used these human oriented tests for their evaluation of GPT4:

Uniform Bar Exam (MBE+MEE+MPT)

LSAT

SAT Evidence-Based Reading & Writing

SAT Math

Graduate Record Examination (GRE) Quantitative

Graduate Record Examination (GRE) Verbal

Graduate Record Examination (GRE) Writing

USABO Semifinal Exam 2020

USNCO Local Section Exam 2022

Medical Knowledge Self-Assessment Program

Codeforces Rating

AP Art History

AP Biology

AP Calculus BC

AP Chemistry

AP English Language and Composition

AP English Literature and Composition

AP Environmental Science

AP Macroeconomics

AP Microeconomics

AP Physics 2

AP Psychology

AP Statistics

AP US Government

AP US History

AP World History

AMC 103

AMC 123

Introductory Sommelier (theory knowledge)

Certified Sommelier (theory knowledge)

Advanced Sommelier (theory knowledge)

Leetcode (easy)

Leetcode (medium)

Leetcode (hard)
A test of your rationality skills
by Max H — 2023-04-20T01:19:03.210Z

[Added April 26: This post has received a fair amount of mixed-to-negative comments and feedback. I'm still not entirely convinced that this isn't a good test, but I retract my request not to comment on the object-level question here or elsewhere.]

This post describes an exercise that tests some aspects of your rationality skills. In particular, it tests for the ability to find and sift through evidence about a complex real-world event, reason probabalistically, draw conclusions, and communicate your reasoning process in writing.

If left unspoiled, it could potentially be used as part of a job application process for a job where these skills are important, such as a research role at an effective altruism or AI alignment organization.

In other words, this is a proposal for a verification method for rationality skills at the organizational level.

The exercise is similar to the Amanda Knox test from over a decade ago. A conclusion and post-mortem from that test are good background reading, though I believe this one is somewhat more difficult and more time-consuming to give a good answer for.

The exercise

In short, figure out what happened during a controversial high-stakes poker game live-streamed in September 2022. In particular, come up with a probability estimate that Garrett Adelstein or other players were cheated, and if so, how and by whom.

Write up your reasoning and your research process, discussing how you weighted different pieces of evidence and why.

More background, from a report on the incident by a consulting firm hired by the casino where the incident occurred:

On September 29, 2022, one of the most controversial hands in poker history was played on an episode of “Hustler Casino Live,” which streams high-stakes poker games played at Hustler Casino in Gardena, Calif., to a worldwide audience on its YouTube Channel. The hand has subsequently been viewed by hundreds of thousands, if not millions, of poker fans around the globe. In the hand, recreational poker player Robbi Jade Lew called an all-in shove by Garrett Adelstein for her remaining $109,000 with the jack of clubs and four of hearts on a board reading ThTc9c3h. Lew ended up winning a pot of $269,000 when her jack high held against Adelstein’s eight of clubs and seven of clubs. After the hand, Adelstein said he was suspicious of Lew’s play and she agreed to return half the pot to him. In the weeks th

...
Language Models are a Potentially Safe Path to Human-Level AGI
by Nadav Brandes — 2023-04-20T00:40:16.102Z

The core argument: language models are more transparent and less prone to develop agency and superintelligence

I argue that compared to alternative approaches such as open-ended reinforcement learning, the recent paradigm of achieving human-level AGI with language models has the potential to be relatively safe. There are three main reasons why I believe LM-based AGI systems could be safe:

They operate on text that is intelligible to humans, which makes them relatively interpretable and easier to monitor.
They are subject to weaker pressure to surpass human-level capabilities than systems trained on more open-ended tasks, such as those pursued by reinforcement learning.
Since language models are trained as predictors, there is weaker pressure for them to develop agentic behavior.

I acknowledge that these arguments have been criticized. I will try to defend these statements, delving into the nuances and explaining how I envision relatively safe LM-based AGI systems. I want to note upfront that although I believe this paradigm is safer than other alternatives I've come across, I still think it poses significant dangers, so I’m not suggesting we should all just chill. It's also difficult to reason about these sorts of things in the abstract, and there's a good chance I may be overlooking critical considerations.

Human-level AGI might be reachable with existing language models

There are

indications

that GPT-4 could be considered an early form of AGI. But even though it surpasses human level over many standardized tests, as a general-purpose AGI it’s not yet at human level, at least in its vanilla form. For example, it’s still not very good at separating facts from fiction, and I wouldn’t give it full access to my email and social media accounts and ask it to throw a birthday party for me.

To address these limitations, there's a project underway that seeks to develop more capable and agentic AI systems based on language models through

chained operations

such as chain-of-thought reasoning.

Auto-GPT

is a very early example of this approach, and I expect to see much better LM-based AI systems very soon. To understand this paradigm, we can think of each GPT-4 prompt completion operation as an atomic operation that can be chained in sophisticated ways

to create deeper and more intelligent thought processes

. This is analogous to how humans engage i...
Alien Axiology
by snerx — 2023-04-20T00:27:08.805Z

I argue that there is a convergent axiological framework by which all intelligent life in the universe trends towards — that there is a universal-like account of the resource constraints, psychology, and game theory of all species across all worlds. I argue for this to help flesh out cross-disciplinary understandings of how value systems work and to draw laws in the study of axiology with which we can make accurate predictions of intelligences we have yet to encounter.

Convergence

Convergent evolution is a phenomenon in biology that selects for otherwise complex biological structures across independent evolutionary tracks. For example, eyes have independently evolved over a 100 different times in myriad environments. This is because light-sensing cells and organs are almost universally advantageous, with the only exception being deep cave systems that don't get any sunlight. So given enough time, complex organs like eyes are expected to evolve as an adaptation of any complex lifeform. This is something we can generalize as a law in exobiology — that by the time you reach higher intelligence you will have already gone through a long evolutionary track that has selected for universal advantages like vision. Just the same, we can expect most highly intelligent alien life to be humanoid, since walking upright uses the most minimal number of limbs for locomotion and conserves the most energy — energy that is needed for metabolically demanding pathways like cognition.

The same line of reasoning leads one to expect a similar convergent phenomenon in exoethology. Behaviors trend towards the social and the cooperative as a species spreads across more environments and becomes more intelligent. From the eusocial insects like ants and bees all the way to humans, the larger the population and the more environments the population spreads into, the more social and cooperative the species needs to be (with members of its own kind) in order to maintain the expansion. The more intelligent a species is (primates, cetaceans, and so on), the more complex their social/cooperative behaviors become (e.g., dolphins play culture-specific games).

The Uncooperative Forest Counter

This sounds like good news since it would mean we could expect any intelligent alien life we encounter to more often than not be humanoid and highly cooperative. But there are obvious counters to this; the book The...
Responsible Deployment in 20XX
by Carson — 2023-04-20T00:24:08.647Z

The purpose of this post is to briefly talk about two categories of alignment and how I think AI safety evaluations should use them. I only consider responsible deployment in 20XX because I often find discussions about 2XXX to be distracting and unproductive. Also, for the remainder of the post, I will only be talking about LLMs, since these could be the 'operating systems' of future AI's (Ex. HuggingGPT). In the near future, LLMs have the highest potential for abuse, and they will be the focus here.

As Connor Leahy and possibly others say, we should be optimistic for the sole reason that we get to build the AIs of the future. A more realistic version of this quote is: we should be optimistic about building our own AI, while acknowledging that others get to build their own AI. This adjustment is made to bring light to the fact that without standardization of safety practices, different groups will have potentially unsound or unknown safety practices.

Safety standardization in AI is challenging due to the lack of comprehensive safety tools, but I want to outline potential industry standards for my own edification. This will be a reminder of the safe practices that I will advocate for during my career.

Mandatory Alignment Evaluations!

Alignment evaluations should be mandatory for all models after GPT-4 and should be conducted by third-party and in-house groups. It will not be enough if AI companies are relied on to graciously open their doors to the Alignment Research Center (ARC). By sheer chance the OpenAI executives had the integrity to have GPT-4 be tested for self-replication and resource acquisition. While the openness is a great present state, my concerns lie with AI capital owners of the near future and the absence of concrete safety obligations, standards and regulations.

The case for oversight and regulation is difficult to make in the United States, but it's an important one. The main crux of the argument for regulation is the classification of generated content as a commercial and industrial product. This definition informs an intuition for the regulation of generated content on the same basis as any influential product massively distributed to the public sphere and private sectors (i.e. food stuff, fabric, children's toys, cyber security etc.). I plan on making a post about that in the future and when I do it will be linked here....
How do I get all recent lesswrong posts that doesn't have AI tag? 
by  — 2023-04-19T23:39:42.637Z

This forum's recent post page is just overloaded with AI content and I don't just want to read curated posts either. I find it strange that their isn't any advanced filtering options. Here is what I want:

input: POSTS(-tag:AI decoupling)

output: all POSTS without the tag AI and the keyword "decoupling" INSIDE

Yes, I suppose I could use search engines and I have done that but it doesn't just search post content but also the content outside of it.
Stop trying to have "interesting" friends
by eq — 2023-04-19T23:39:25.575Z

Originally published on substack: https://eliqian.substack.com/p/stop-trying-to-have-interesting-friends

Within the tech and tech-adjacent circles I’m part of on the internet, a number of essays and tweets have been making the rounds about what makes someone a person you want to spend time with.

This is definitely a worthwhile question to answer. So much of our life experience is colored by the people we share it with. However, there is a concerning amount of emphasis on having friends and seeking out people who are smart, thoughtful, and (the worst one of all) “interesting.”

The obsession with interesting makes me uneasy. Friends aren’t resources for intellectual stimulation or new insights. I don’t want my friends to like me because I read niche blogs or have things to say about crypto. It comes dangerously close to conflating knowing a lot, reading a lot, or having thoughtful things to say with moral goodness. (Worth noting that interesting isn’t a negative trait. I would prefer my friends to be interesting, but it feels superficial to prioritize it.)

If I think about what kind of person I want to be close friends with—my best man at my wedding, people I would do anything for, the ones I can rely on when I’m at my lowest—smart and interesting really aren’t front of mind.

On top of that, it doesn’t feel right to pressure people to be “interesting” in your sense of the word. Everyone is interesting. Every person in the world has a literal lifetime of experiences that have shaped who they are. They have internal thought processes and distinct worldviews that you won’t find within anyone else. It’s a matter of giving your interactions enough time and care to discover these.

You shouldn’t feel obligated to be friends with everyone or pressured to get along with all people, but “interestingness” is a poor heuristic for finding genuine connection. It’s time we realign our priorities and recognize that our preoccupation with being interesting is symptomatic of a flawed view of friendship.

Ava of Bookbear Express

made an observation

I think about often:

It’s not about what you find intellectually cool, or what seems like the best “opportunity.” Those things can be important too, but they don’t matter if you hate doing the thing…

You have to do the thing you actually enjoy doing, not the thing you find conceptually exciting.

It’s easy to convinc...
Is there any literature on using socialization for AI alignment?
by Nathan1123 — 2023-04-19T22:16:36.578Z

Hello,

I was recently thinking about the question of how humans achieve alignment with each other over the course of our lifetime, and how that process could be applied to an AGI.

For example, why doesn't everyone shop lift from the grocery store? A grocery store isn't as secure as Fort Knox, and if one considered every possible policy that results in obtaining groceries then they may discover that shop lifting is more efficient than obtaining money from a legitimate job. That may or may not be the best example, but I'm sure LW is quite familiar with the concept that lies at the heart of the problem of AI alignment: what humans consider the morally superior solution isn't always the most "rational" answer.

So why don't humans shop lift? I believe the most common answer from modern sociology is that we observe that other humans obtain jobs and pay with legitimate money, so we imitate that behavior out of a desire to be a "normal" human. People are born into this world with virtually no alignment, and gradually construct their own ethical system based on interactions with other people around them, most importantly their parents (or other social guardians).

Granted, I'm sure from the perspective of ethical philosophy and decision theory that explanation could be an oversimplification, but my point is that socialization would appear to be a straight-forward solution towards AI alignment. When human beings become grown adults, and their parents are considerably weaker from old age, then their elders no longer have any physical capability of controlling them. And yet, people obey or respect their parents anyway, and are expected to morally speaking, because of the social conditioning they still recall from back when they were children. That is essentially the same outcome we want to have with a Superintelligent AGI: a being that is powerful enough to ignore humanity, but has a deep personal desire to obey them anyway.

Some basic mechanics of formal and informal norms in sociology could lend themselves towards reinforcement learning algorithms. For example:

Guilt-based discipline: as the AGI explores its environment, indicate when a state-action pair of an adopted policy is morally wrong

Shame-based discipline: whenever the AGI adopts a policy that has a detrimental outcome, indicate that its general behavior is morally wrong

One possible criticism of socialization al...