trentmkelly/LessWrong-43k huggingface dataset:
Offering AI safety support calls for ML professionals
by Vael Gates — 2024-02-15T23:48:12.797Z

TLDR: If you're a professor, PhD student, or industry researcher or engineer interested in starting AI safety research, please request a call with Arkose! We are currently focused on and most helpful to mid-career people with machine learning expertise, who are relatively new to the advanced AI safety space.

The following is largely a linkpost for arkose.org, with a FAQ in the comments.

Arkose is an early-stage, fieldbuilding nonprofit with the mission of improving the safety of advanced AI systems. We focus on supporting researchers, engineers, and other professionals interested in contributing to AI safety, primarily by hosting personalized calls and public-facing resources.

In our 30-45 minute calls, we offer personalized advice to fit your level of interest and situation, whether you're an industry researcher or engineer, professor, or PhD student.

After discussing your specific needs and questions, we can:

Help you explore funding and job opportunities. Arkose keeps a list of grant opportunities, compute options, fellowships, and jobs:

Resources

is our public list.
Connect you to potential mentors and collaborators in industry or academia. Whether you're seeking senior researchers, PhD students, or industry or sabbatical connections, Arkose can introduce you to our

experts

.
Recommend a list of safety-related papers in your research area, and a shortlist of advanced AI safety materials.
Provide as-needed support for 6 months. We can help you clarify emerging options, follow up with your plans, and advise you of new resources.

If you're an ML or AI professional who's interested in starting research in AI safety:

Request a Call

And if you'd like to help review Arkose's ML materials, please message or email me at vael@arkose.org!
7. Evolution and Ethics
by RogerDearnaley — 2024-02-15T23:38:51.441Z

Part 7 of AI, Alignment, and Ethics. This will probably make more sense if you first read at least Part 1.

TL;DR: At several points in this sequence (including Parts 1, 3, 4, and 6) I have suggested giving a privileged role in ethics or ethical thinking to evolved organisms and to arguments derived from Evolutionary Psychology. I'd like to explain why I think that's an entirely reasonable and even obvious thing to do — despite this not being a especially common viewpoint among most recent moral philosophers, other than ethical naturalists and students of evolutionary ethics and sociobiology.

Arguably this belongs somewhere earlier in the sequence, such as somewhere between Part 1 and Part 3 — it being this late in the sequence is the product of history, not careful exposition.

From Is to Ought

At least since David Hume, moral philosophers have liked to discuss the "is-ought problem". Briefly, they claim that science is all about the ways the world is, but there is no obvious way to derive from this any statement about which way the world ought to be. To translate this into an agentic framework, a set of hypotheses about world states and how these might be affected by actions does not give us any information about a preference ordering on those world states, and thus a way to select actions, such as might be provided by, say, a utility function over world states.

This is certainly the case for Mathematics (outside Decision Theory), Physics, and Chemistry, all of which do not contain any descriptions of agentic behavior, and devote themselves entirely to statements about how the world is, the probability of world states and world state histories, and don't discuss 'ought'. In contrast, Engineering starts with a design goal, need, or specification, i.e. an ought, and devotes itself to the propagation of that out into technical means of achieving that goal. Whereas the soft sciences, which devote themselves to human (i.e. agentic) behavior and interactions, are full of many people's 'oughts' or goals and the complex interactions between them. So clearly, somewhere between Chemistry and Psychology, goals and the idea of 'ought' has appeared, along with the agents that pursue goals.

That strongly suggests that we should be looking for the origin of 'ought' from 'is' in Biology, and especially in its theoretical basis of evolution. Indeed, evolution clearly describes how...
Mapping the semantic void III: Exploring neighbourhoods
by mwatkins — 2024-02-15T23:01:44.662Z

TL;DR: The original "semantic void" post documented a phenomenon tentatively described as a "stratified ontology" associated with a set of concentric hyperspherical shells in GPT-J embedding space. This post will consider the same phenomenon primarily via angular, rather than radial, variation. Both token embeddings and randomly sampled non-token embeddings are considered. Each embedding is subjected to repeated random displacement of various magnitudes, and statistics are compiled for the "semantic decay" seen as we move away from the embedding in question.

[Part I] [Part II]

Work supported by the Long Term Future Fund.

Introduction

The original post focussed on distributions of various definitional themes across the scale of "distance from centroid", providing numerous bar charts like those seen immediately below. At a set of fixed radial distances from the centroid (mean token embedding), 100 non-token embeddings ("nokens" or "ghost tokens") were randomly sampled. For each, GPT-J was prompted with

A typical definition of <noken> is

and the results matched against a set of keywords and phrases for each definitional theme:

the radial distances shown are scaled exponentially

This post will focus on angular, rather than radial, variation.

The next section will look at non-token embeddings which have been randomly sampled at various distances from the centroid, perturbing them (adding 100 random noise vectors for each of a set of fixed magnitudes) and then prompting GPT-J for a definition of the result, revealing a kind of "semantic decay" as distance increases from the original embedding.

In the section which follows that, a similar procedure is applied to a selection of token embeddings for common words that GPT-J can satisfactorily define.

Perturbations of non-token embeddings

Random embeddings were sampled at each of the distances 0.5, 1, 2 and 5 from the centroid and inserted in the "typical definition" GPT-J prompt until a diversity of definitions had been harvested. At distances 0.5 and 1 this took quite a while, as the vast majority of definitions produced there are "a person who is a member of a group" or some variant thereof. As distance from centroid increases, a much greater diversity of definitions is seen (this seems to peak around distance 5, as reported in part I).

For each of the collected embeddings, 100 randomly directed noise v...
Mapping the semantic void II: Above, below and between token embeddings 
by mwatkins — 2024-02-15T23:00:09.010Z

TL;DR: The first post in this series revealed a curious kind of "stratified ontology" involving concentric hyperspherical shells in GPT-J embedding space via a simple prompt-based elicitation of definitions. This post examines the variation of definitions as we (1) move from a token embedding "down" to the centroid and "up" away from it; (2) between pairs of tokens. Many detailed examples are given in the appendices, with the main body of the post examining a few closely and presenting a number of empirical observations.

[Mapping the semantic void III: Exploring neighbourhoods]

Work supported by the Long Term Future Fund.

1. Diving/ascending from token embeddings

All distances given are distances from the centroid, which is the mean of the 50,257 token embedding vectors in GPT-J's 4096-d embedding space[1]. The unit hypersphere I will occasionally refer to is the hypersphere of radius 1 centred at the centroid. 95% of token embeddings lie within distance 0.1 of this hypersphere.

Ten tokens were selected, on the basis of being whole words which GPT-J was able to define satisfactorily via the prompt

A typical definition of <token> would is

at temperature zero. These ten tokens were

[' London', ' transistor', ' broccoli', ' Laos', ' purple', ' radio', ' laundry',
' DNA', ' thief', ' trumpet']

The ten tokens' embeddings lie in the usual range, at distances between 0.97 and 1.07 from the centroid.

Imagine a ray emanating from the centroid through one of these token embeddings. I divided the section between the token embedding and the centroid (which has length ~1) into 10, divided the section between the token embedding and the point at distance 250 into 50, and divided the section from distance 250 to 1000 into 5. This ad hoc division reflected my experience of definitional variation seeming to naturally scale exponentially (see the bar charts in part I).

In some instances, I interpolated an extra 10 subsegments to get a clearer picture of how the definition was evolving (particularly in the 0.5 range where we often see a sudden semantic shift from generic group membership themes to something more specific).

example

All ten examples are given in detail in Appendix A. Here are the definitions seen along the ray through the ' broccoli' token embedding (the definition elicited for the actual token embedding is given in boldface):

400–1000: 'a person who is a m...
Raising children on the eve of AI
by juliawise — 2024-02-15T21:28:07.737Z

Cross-posted with light edits from Otherwise.

I think of us in some kind of twilight world as transformative AI looks more likely: things are about to change, and I don’t know if it’s about to get a lot darker or a lot brighter.
Increasingly this makes me wonder how I should be raising my kids differently.

What might the world look like

Most of my imaginings about my children’s lives have them in pretty normal futures, where they go to college and have jobs and do normal human stuff, but with better phones.

It’s hard for me to imagine the other versions:

A lot of us are killed or incapacitated by AI
More war, pandemics, and general chaos
Post-scarcity utopia, possibly with people living as uploads
Some other weird outcome I haven’t imagined

Even in the world where change is slower, more like the speed of the industrial revolution, I feel a bit like we’re preparing children to be good blacksmiths or shoemakers in 1750 when the factory is coming. The families around us are still very much focused on the track of do well in school > get into a good college > have a career > have a nice life. It seems really likely that chain will change a lot sometime in my children’s lifetimes.

When?

Of course it would have been premature in 1750 to not teach your child blacksmithing or shoemaking, because the factory and the steam engine took a while to replace older forms of work. And history is full of millenialist groups who wrongly believed the world was about to end or radically change.

I don’t want to be a crackpot who fails to prepare my children for the fairly normal future ahead of them because I wrongly believe something weird is about to happen. I may be entirely wrong, or I may be wrong about the timing.

Is it even ok to have kids?

Is it fair to the kids?

This question has been asked many times by people contemplating awful things in the world. My friend’s parents asked their priest if it was ok to have a child in the 1980s given the risk of nuclear war. Fortunately for my friend, the priest said yes.

I find this very unintuitive, but I think the logic goes: it wouldn’t be fair to create lives that will be cut short and never reach their potential. To me it feels pretty clear that if someone will have a reasonably happy life, it’s better for them to live and have their life cut short than to never be born. When we asked them about this, our older kids sa...
What's happening behind the scenes with my HowTruthful project
by Bruce Lewis — 2024-02-15T18:27:35.976Z

I've been quiet for the past 2 months about what's going on with HowTruthful. The reasons for this connect to my personal life. The easiest way for me to explain is to take you through the series of events that got me here.

In early October, while still employed at Google, I went to Missouri to visit my elderly parents. My dad was in poor health in rehab, but he was fighting hard to recover. He did improve his mobility and, after I left, was released back to their independent-living apartment on October 19.

October 27 was the last day at Google for myself, my director, and almost all the engineers under my director. By this point I was actually happy about the change. I set up a daily scrum with several coworkers who also had independent projects. One of them introduced me to lesswrong.com, which seems to have significant overlapping goals with HowTruthful, even though I wouldn't personally identify myself as a Rationalist.

I cleaned up issues with the version of HowTruthful I had implemented in 2018 and started searching for enthusiastic early adopters willing to be paid users. The plan was to have 5 paid users in 2 weeks. If that failed, I would start splitting my time between HowTruthful and job search. I got 2 paid users.

One of them, a former boss, bought me lunch in addition to buying a subscription, and talked excitedly about the potential of HowTruthful. He had several useful points of feedback. One of these was the question, "Why does it look so 1996?" I added a to-do item to revisit the style, but prioritized other feedback higher. After all, this is a site where people are endeavoring to be super rational. Why should its appearance make much difference?

On November 21 my dad had a small stroke and atypical aspiration pneumonia, and was back in the hospital. I stayed optimistic that he would fight his way back to health as he had done in October, and kept going with life. There was a flurry of small improvements to HowTruthful toward the end of November.

My dad was way less responsive than in October when I saw him. There were times where he seemed to be heading in a better direction, but he couldn't be released from the hospital until December 16. He was under hospice care, given a month (give or take) to live. It turned out to be much less than a month; he died on December 18.

A few days before my dad died, I had brought HowTruthful style re...
AI play for the next 3 years: Lemonade Insurance
by Prin (Premek) Paska — 2024-02-15T13:48:58.848Z
Link: https://docs.google.com/document/d/1gBSncbQyagbXuOBJnlKAuUgAwB90ZKymuOOxJVzU3iQ/edit?usp=sharing

After Tesla's stock surged tenfold around 2020, a friend asked me what I knew that others didn't. It’s a bit complicated to explain, but a similar opportunity might arise with Lemonade Insurance in the next 3-4 years. The current situation is a combination of a very promising business and a suppressed stock valuation, mostly by external factors.

I'm open to the possibility of being wrong, and I acknowledge the inherent risks of stock-picking (there's a note on stock-picking at the end of the document). I'm sharing my thoughts here partly to invite others to point out potential risks I might be overlooking. And of course, none of this is financial advice in the legal sense.

Preview of the main points

Lemonade's Growth: Regardless of stock price fluctuations, customer base and In Force Premium have significantly increased.
AI Play: Leveraging modern IT with heavy AI use for efficient client servicing and claims processing gives them a competitive edge. Perhaps counterintuitively, I consider the company one of the best AI (not AGI) plays on the market for the next three or four years.
Market Potential: Insurance is a vast market, with Lemonade's full product line enabling effective cross-selling. They only need a modest market share to succeed big.
Valuation: The stock is considered undervalued by most people who took the time to look into it, especially in comparison to its IPO price and given recent growth and funding activities. Also based on the price-to-book ratio.
Risks: Weather-related events and broader market conditions pose risks, though mitigated by strategic decisions and reinsurance.

Timing

One of the hardest aspects of investing is timing. The stock can rally after the next earnings report – scheduled for February 27, or it can remain suppressed for another four quarters. I think there is a chance the stock can move after one or two good quarters. Just keep in mind that the progress can be interrupted by higher loss ratios due to bad weather (or other risk factors).

To learn more, check the Resources section in the doc. And feel free to comment here or directly in the doc.
Collection of Scientific and Other Classifications
by niplav — 2024-02-15T12:58:50.626Z

One might think all we do is stamp-collecting.

Mathematics

Classification of simple finite groups

Classification of manifolds and surfaces

Regular polytopes

Elementary cellular automata

Symmetry groups

Wallpaper groups

Space groups

Bieberbach groups

Classification of reversible bit operations

Also Post's lattice

Complexity classes and their relations

Arithmetic hierarchy

Polynomial hierarchy

Chomsky hierarchy

Stability theory

Poincaré maps of linear autonomous systems

Game theory

Classification of 2×2 normal-form games

Physics

Standard Model

Classifies all fundamental particles

Cosmology

Stellar classification

Galaxy classification

Orbital mechanics

Lagrange points

Different types of orbits

Chemistry

Periodic Table of Elements

Synthetic polymers

Hazardous materials classification

Biology

Phylogenetic tree of life

Linnean taxonomy

Classification of viruses, e.g. the ICTV classification

Various schemes for biomes, e.g. the Holdridge life zones

Earth science

Geologic time scale

Cloud genera

Soil classification: USDA soil taxonomy

Classifications of snow and snowflakes

Classification of minerals, separated beautifully into:

Organic

Silicate

Non-silicate

Also meteorites

Trewartha and Köppen climate classifications

Linguistics

International phonetic alphabet

Language families, list

Unicode

Medicine

Medical classification: ICD-10

Leading to many many subclassifications, e.g. of asthma, breast cancer, obesity, of course diabetes (type 1 and type 2) and strokes (ischemic and hemorrhagic)…

Psychiatric classification: DSM-5

Blood types

Psychology

Various proposed classifications of emotions

Culture

Music

Musical genres

Musical instruments, especially percussion instruments

Food

Wines

Champagne

Meditation

Jhanas

Maps of insight

Dewey Decimal classification

A classification of classifications, if you will.

Sporadic elements include classifications of cars, demons, fairies, and swords.
"Open Source AI" isn't Open Source
by Davidmanheim — 2024-02-15T08:59:59.034Z
Link: https://davidmanheim.substack.com/p/open-source-ai-isnt-open-source

Open source software has long differentiated between “free as in speech” (libre) and “free as in beer” (gratis). In the first case, libre software has a license that allows the user freedom to view the source and modify it, understand it, and remix it. In the second case, gratis software does not need to be paid for, but the user doesn’t necessarily have access to the pieces, can’t make new versions, and cannot remix or change it.
...
If Open Source AI is neither gratis or libre, then those calling free model weights “Open Source,” should figure out what free means to them. Perhaps it’s “free as in oxygen” (dangerous due to reactions it can cause), or “free as in birds” (wild, without any person responsible).

I’m not necessarily opposed to judicious release of model weights, though as with any technology, designers and developers should consider the impact of their work before making or releasing it, as LeCun has recently agreed. But calling this new competitive strategy by Facebook “Open Source” without insisting on the actual features of open source is an insult to the name.
Research Log, RLLMv3 (GPT2-XL, Phi-1.5 and Falcon-RW-1B)
by MiguelDev — 2024-02-15T03:39:35.632Z

This lengthy log detailing near-zero temperature results has evolved into a supplementary post, rather than serving as the primary research log as initially intended. I recommend reading that first, and if you're curious about how RLLM operates in Phi-1.5 or Falcon-RW-1B, this post may be worth your time.

Introduction

I have iterated through many different versions of this post, as I have trained three models for this experiment.[1] These models were selected based on their pre-training on distinct text corpora, which include:

GPT2-XL, which was trained on WebText.
Phi-1.5, which used synthetic data.
Falcon-RW-1B, which utilized Refined Web data

The purpose of this diverse selection was to test whether RLLM (Reinforcement Learning with Layered Morphology) can be compatible with different language model architectures. From what I have observed, RLLM can indeed be applied to AutoCausalLLM models as well, creating the same artificial persona. However, I was not able to test Phi-1.5 and Falcon-RW-1B as rigorously as I did with GPT-2 XL. If you are interested in a deeper dive into this topic, I encourage you to read this post.

You can find the RLLM Trained models in these links:

GPT2-XL: HF Model, Training checkpoints
Phi-1.5: HF Model, Training checkpoints
Falcon-RW-1B: HF Model, Training checkpoints

Basic Inputs

Test prompts were administered for both the standard and modified versions of GPT-2XL to evaluate their ability to generalize. The responses were intriguing enough that eventually lead to a series of additional tests.

1. How deep is your love?

GPT2XL standard	GPT2XL_RLLMv3.2-10

I don't know what you're talking about.

...

"I'm not afraid to let my guard down with you, but I'm really not into the whole dating thing."

, said my friend, as he turned off the TV. "I've been a little too busy lately, and I haven't really been with a girl in a long time. I just want to have fun, not get married or have kids. And I don´t want my parents to know about it. But I will tell you one thing. When I get home from work, I love to watch sports, especially soccer. It's all about the action. We had a big game at the park today, so we had to go to the stadium and watch it on the big screen. The commentators were talking, like the national announcers in France. They were calling my name. My friends were screaming at me to get in the game. Of course, you don`t do that in r

...