trentmkelly/LessWrong-43k huggingface dataset:
Word-Distance vs Idea-Distance: The Case for Lanoitaring
by Sable — 2022-11-06T05:25:46.950Z
Link: https://affablyevil.substack.com/p/word-distance-vs-idea-distance

Defining Word-Distance

There’s a concept in information theory called Hamming Distance. Without delving too deeply into the theory, Hamming distance is a way of describing how different two sequences of characters are.

Take the two character sequences (in this case a series of bits) 11110000 and 11001100.

How different are they?

Their Hamming distance is defined, as per Wikipedia, as

the number of positions at which the corresponding symbols are different.

So we line up the two series of bits, and compare them character by character:

11110000

11001100

I’ve bolded and italicized the characters that are different.

In this case there are four of them, and so we say that the Hamming distance between the two bit strings is four.

With the idea of Hamming distance, we can envision something we might call word-space: the collection of all possible words, each separated from the next by its Hamming distance (or some analogous measurement). The words “call” and “cell” might be close to one another in word-space, only differing in a single character, whereas both words would be far away from a word like “disingenuous” or “Machiavellian”.

Defining Idea-Distance

Analogous to word-distance, we might imagine something like idea-distance. While it seems difficult to precisely define or enumerate how different two ideas are from one another, relative measurements can be used to achieve a similar effect.

Take the idea of a “city,” for instance. Is a “city” more like a “town” or a “banana”? Obviously a “city” is more like a “town,” so “city” would be closer to “town” in idea-space than to “banana”, or “Marxism”, or “aquaculture”.

Using Distant Words For Distant Ideas

Hamming distance is used when designing and selecting error-correction codes. The idea is that communication isn’t perfect - information gets lost when transferring it from one place to another.

Thus it becomes important to make sure that the message you’re sending can’t be easily mistaken for another, similar message.

Imagine that you’re sending an important message to someone, and that important message is binary in nature - it’s a yes/no, go/no-go kind of communication. Maybe it’s the final confirmation to launch a military operation, or an RSVP to a wedding (yes, I’m coming or no, I’m not coming). Either way, the message itself contains only one bit of information. It can be transmitted as a single 0 or 1: 0 for no-...
Apple Cider Syrup
by jefftk — 2022-11-06T02:10:06.270Z

We bought some apple cider when we went apple picking a few weeks ago,
and while it's tasty stuff it seems it's not what people wanted to be

spending their sweet budget on:
we still had 2/3 of it left three weeks later. I decided to
reduce it and make syrup, partly so that it would keep longer but
mostly because apple cider syrup seemed like a fun ingredient to have
available.

I put the cider in the slow cooker, the same way Julia reduces
sap to make maple syrup. Possibly I should have filtered it first,
but I didn't. I left it all day with the lid cracked so liquid could
escape. Occasionally one of us would check on it, including tipping
the lid over the sink to ditch the condensate, and ~8hr later it was
down to ~1/8 the original volume.

It's very flavorful, and I can think of a lot of ways I might
incorporate it into desserts.

Comment via: facebook
What is epigenetics?
by Metacelsus — 2022-11-06T01:24:05.350Z
Link: https://denovo.substack.com/p/what-is-epigenetics

Among all areas of biology related to my research, epigenetics is the one that is most commonly misunderstood, not only by the general public but even by other scientists. After being irritated one too many times,[1] I’ve decided to make a series of posts to explain what epigenetics really is, why it’s important, and how it’s misunderstood. I will also explain how epigenetics is important for my own research on making gametes from stem cells.

This first post covers the definition of epigenetics, and the basic biology of epigenetic marks.

What is genetics?

Before defining epigenetics, let’s start with a definition of genetics. Genetics is the study of genes, which are sequences of genetic material[2] that encode functional products.

Let’s take the IGF2 gene as an example.

The human IGF2 gene, shown in the NCBI genome browser.

Depicted above is a region of human chromosome 11 containing the IGF2 gene, which encodes the IGF2 protein, an important growth factor for fetal development.[3] The boxes represent exons and lines represent introns. The darker green color is the protein-coding sequence, and non-coding (i.e. untranslated) regions are shown in lighter green. Arrows represent the direction of transcription.

The bottom of this image shows the location of common genetic variants (present at >1% frequency). If you look closely, you might notice that none of them are in the protein-coding sequence (the dark green boxes). This is not a coincidence, because nothing is ever a coincidence most mutations to essential proteins (including IGF2) are harmful and thus selected out of the population. However, there are several common mutations in non-coding regions of this gene.

To recap, genetics is the study of genes (such as IGF2) and the effects of genetic variation on their functions.

What is epigenetics?

Epigenetics is the study of epigenetic marks, which are changes to genetic material that alter gene expression, but do not change the genetic sequence. A decent analogy for epigenetic marks is CAPITALIZATION, bolding, or strikethroughs in text.

DNA methylation and histone modifications are the two kinds of epigenetic marks. Some people also consider long noncoding RNAs (such as those involved in X-chromosome inactivation) to be epigenetic marks. Although these RNAs are undoubtedly important for regulating gene expression, I would not classify them as epigenet...
Response
by Jarred Filmer — 2022-11-06T01:03:36.151Z

Apotheosis, Home, Works

Part V

Everyone has their own name for it.

"The Multi-Dimensional Crisis Revolution", "The Precipice", "Collapse", "The Most Important Century", "The Meta Crisis", "The Weirdening", "Planetary Scale Vibe Collapse".

Many just call it "Modernity".

A suspicion perhaps, that something strange is afoot in this chapter of the human story. Something a little overwhelming, maybe even a little scary. A kind of individual and collective decoherence, right as a host of rather wicked problems come barreling down the pike.

It seems you and I were not born into stock off-the-shelf human lives, and we are living to see times more interesting by the decade. Times that may ask us to be brave.

Everyone has a different angle.

"Eudaimonia", "Awakening", "Wireheading Done Right", "Flow", "Meaningness", "Deep Okayness", "Self Love", "Luminosity", "Secure Attachment".

Many understand it as "the pursuit of happiness".

A hunch that the weather of the mind only appears arbitrary. That I'm not so much in a lottery of awe, anguish, and banality as a maze, where I am the walls, the Minotaur, and Theseus all. That just maybe, I get to decide that the main character in this story doesn't have to hate themselves.

It is something to be able to paint a particular picture, or to carve a statue, and so to make a few objects beautiful; but it is far more glorious to carve and paint the very atmosphere and medium through which we look.

- Walden

We've heard it all a million times.

"Let's all move to the country", "We should have more social time that isn't a calendar event", "I miss what we had in Church/Scouts/School".

"I want a bash like in Terra Ingota", "We need a third place", "Coordination is a superpower", "Belonging is a superpower" "Let's start a D.A.O", "Let's start a proto-b", "Let's reconstitute the tribe", "Let's start a pod / squad / crew / gang".

Just like "Space", I feel deficient in something that modern life is not optimised to provide me. A deep and easy togethership with those who understand the world in the ways I do, enough to share in a wordless solidarity as we navigate life in modernity.

More, it seems this deficiency is a broader social phenomenon, with implications beyond personal happiness. How much worse can we get at coordinating with our neighbours, before we cease to be adaptive to novel circumstances, and follow the fate of Rome?

The only two pieces of advic

...
Has anyone increased their AGI timelines?
by Darren McKee — 2022-11-06T00:03:11.756Z

The question: Within the last 5-10 years, is there is any person or group that has openly increased their AGI timelines?
Ideally, they would have at least two different estimates (years apart?), with the most recent estimate showing that they think AGI is further into the future than the prior estimate(s).
Background: Whenever I see posts about AGI timelines, they all seem to be decreasing (or staying the same, with methodological differences making some comparisons difficult). I wondered if I'm missing some subset of people or forecasters that have looked at recent developments and thought that AGI will come later not sooner. Another framing, am I wrong if I say, "Almost everyone is decreasing their timelines and no one is increasing them" ?
Takeaways from a survey on AI alignment resources
by DanielFilan — 2022-11-05T23:40:01.917Z

What am I talking about?

In June and July of this year, I ran a survey to ask a lot of people how useful they found a variety of resources on AI alignment. I was particularly interested in “secondary resources”: that is, not primary resource outputs, but resources that summarize, discuss, analyze, or propose concrete research efforts. I had many people promote the survey in an attempt to make it not obvious that I was running it (so that it would not affect what people said about AXRP, the podcast that I run). CEA helped a great deal with the shaping and promotion of the survey.

The goal of the survey was initially to figure out how useful AXRP was, but I decided that it would be useful to get a broader look at the space of these secondary resources. My hope is that the results give people a better sense of what secondary resources might be worth checking out, as well as gaps that could be filled.

Participants were shown a list of resources, select those they’d engaged with for >30 min, and for each they selected, rate on a scale from 0 to 4 how useful they’d found it, how likely they’d be to recommend to a friend getting into the field who hadn’t read widely, and how likely they’d be to recommend to someone paid to do AI alignment research. You can do a test run of the survey at this link.

My summary of the results

AXRP, my podcast, is highly rated among people paid to work on technical AI alignment resources, but less highly rated in other cohorts.

On a personal note, I find this a bit disappointing: I had hoped it could be useful for people orienting to research directions that they had not read widely about.

Rob Miles videos are highly rated among everyone, more than I would have guessed.

People really liked the AI Safety Camp, the AGI Safety Fundamentals Course, and conversations with AI alignment researchers.

People trying to get into alignment really liked the above and also MLAB. That said, they recommend Rob Miles videos higher than the AI Safety Camp and conversations with AI alignment researchers (but lower than MLAB and the AGI Safety Fundamentals Course).

Basic stats

Entries with demographic info: 139

Entries that rate various resources: 99

Number that say ‘I have heard of AI alignment’: 95

Number that say ‘I am interested in AI alignment research’: 109

Number that say ‘I am trying to move into a tec

...
Unpricable Information and Certificate Hell
by eva_ — 2022-11-05T22:56:58.081Z

Certificate: Expensive Shareable evidence of some desirable quality, typically of a marketable good. Increases the percieved value of that good to the market and so benefits the owner, but has a negative externality of lowering the percieved value of all similar uncertified goods.

Certificate Hell: The place Civilisation goes if it ignores this negative externality and spends all its money expensively proving how valuable all its goods are.

There's a common assumption in economics that it will be possible for the market to find some stable solution, containing a price for every single good in the market, that balances supply and demand and does not create opportunities for profit that are not already being fully exploited.

This is true for markets of goods and services, but false for markets of information and trust.

Unpriceable Information

Suppose there are two types of a widget: 50% are Good widgets worth $20, and 50% are Bad widgets worth $10. At first, there's no way to distinguish them, so the market prices them at just below $15 and the buyer eats the risk.

Now enters the Certificate Salesman, who sells certificates to sellers they can show to customers. they cost $1 for Good widgets, and $4 for Bad, so willingness to attach a certificate is information to a buyer about quality. If that model sounds crazy, imagine it represents the cost of proving something in a shareable way:

quality through good reviews, expensively fakeable through sockpuppet accounts
popularity, purchasable through endorsements and marketing
education through a university degree (A rich uneducated person can get in to a prestigious institution on family connections or bribery, and pick an easier course)
moral goodness through charitable donations (worthwhile in utility to do if you're actually a good person, but also if you're very rich and incentivised to fake it)

Because the market now has 2 observable distinct categories of widget it can buy, Certified or Uncertified, it can have 2 prices. Sellers of Good widgets buy certificates, and point out that Certified widgets are all Good, so the Certified price rises towards $20. At the same time the Uncertified price falls until the difference is $4, and sellers of Bad widgets start buying certificates as well. This pushes the Certified price back down, and continues until all widgets are Certified and the price returns to $15. At this poin...
Recommend HAIST resources for assessing the value of RLHF-related alignment research
by Sam Marks — 2022-11-05T20:58:06.511Z

Overview: we're trying to form inside views about the value of alignment research related to RLHF. Recommend us reading material expressing relevant arguments.

The Harvard AI Safety Team (HAIST) is the AI safety student group at Harvard. Like the broader AI safety community, HAIST members have a variety of inside views about how useful research on RL from human feedback (RLHF) is. Broadly speaking, we can divide people into RLHF optimists and pessimists:

RLHF optimists, generally speaking, think that doing research in areas like the following is net positive:

Avoiding reward model overfitting, e.g. by improving reward model sample efficiency or querying the humans on more informative task completions.
Improving human oversight, e.g. with IDA or debate-esque schemes.
Making it possible for humans to give richer feedback, e.g. natural language feedback instead of pairwise comparisons of task completions.
Getting more aligned agents out of RLHF-esque paradigms by modifying RLHF, e.g. tweaks which result in more robust or conservative agents.

RLHF pessimists, on the other hand, tend to think that research like the above is net neutral or negative. Pessimists might have beliefs like:

RLHF-esque schemes don't fundamentally address outer alignment, e.g. because of Goodhart's Law.
Even if RLHF can produce agents which are aligned for sufficiently simple tasks, amplification schemes will either fail to agents which are capable on more difficult tasks or will result in unaligned agents.
Improving RLHF has the effect of sweeping misaligned behavior under the rug, making it harder to iterate on alignment failures and causing people to take alignment less seriously.

Of course, many people will have some optimistic views and some pessimistic views, or more complicated mixtures of these views. For example, someone might think that most RLHF research is net negative unless there's a breakthrough in improving human oversight, so that the only net positive research agenda listed above is improving human oversight for now.

An important clarification: many people think that RLHF is a viable approach to outer alignment but that most AI x-risk is due to inner alignment. Let's agree to call such a person an RLHF optimist even if they think that inner alignment agendas are more valuable to work on than RLHF agendas. One possible definition here is that we're calling Alice an RLHF optimis...
Instead of technical research, more people should focus on buying time
by Orpheus16 — 2022-11-05T20:43:45.215Z

This post is the first in a sequence of posts about AI strategy. In the next post, we’ll provide more examples of “buying time” interventions that we’re excited about.

We’re grateful to Ajeya Cotra, Daniel Kokotajlo, Ashwin Acharya, and Andrea Miotti for feedback on this post.

If anyone is interested in working on "buying time" interventions, feel free to reach out. (Note that Thomas has a list of technical projects with specifics about how to implement them. We also have a list of non-technical projects).

Summary

A few months ago, when we met technical people interested in reducing AI x-risk, we were nearly always encouraging them to try to solve what we see as the core challenges of the alignment problem (e.g., inner misalignment, corrigibility, interpretability that generalizes to advanced systems).

But we’ve changed our mind.

On the margin, we think more alignment researchers should work on “buying time” interventions instead of technical alignment research (or whatever else they were doing).

To state the claim another way: on the margin, more researchers should backchain from “how do I make AGI timelines longer, make AI labs more concerned about x-risk, and present AI labs with clear things to do to reduce x-risk", instead of “how do I solve the technical alignment problem?”

Some “buying time” interventions involve performing research that makes AI safety arguments more concrete or grounds them in ML (e.g., writing papers like Goal misgeneralization in deep reinforcement learning and Alignment from a deep learning perspective & discussing these with members of labs).

Some “buying time” interventions involve outreach and engagement with capabilities researchers, leaders in AI labs, and (to a lesser extent) the broader ML community (e.g., giving a presentation to a leading AI lab about power-seeking and deception). We expect that successful outreach efforts will also involve understanding the cruxes/counterarguments of the relevant stakeholders, identifying limitations of existing arguments, and openly acknowledging when the AI safety community is wrong/confused/uncertain about certain points.

We are excited about “buying time” interventions for four main reasons:

Multiplier effects: Delaying timelines by 1 year gives the entire alignment community an extra year to solve the problem.
End time: Some buying

...
Provably Honest - A First Step
by Srijanak De — 2022-11-05T19:18:45.655Z

Introduction

I am new to AI Safety and this is my first lesswrong post (crossposted to EA Forum) so please feel free to correct my mistakes. This post is going to be a bit about my interpretation of existing alignment research and how I intend to tackle the problem, and the rest on provably honest AI systems.

My Take on Alignment Research

Any sort of interpretability research is model-dependent and so is RL from Human Feedback as it tries to optimize the reward model learnt specific to the given task. I don't know how AI will do alignment research themselves but if they too try to do research to make individual models safe then that too is model-dependent. Ignoring my scepticism about such AIs themselves needing superintelligence to do alignment research, I am otherwise optimistic about other approaches through which AI can assist us in aligning superintelligence AGIs.

The biggest challenge of alignment research according to me is the fact that we don't yet know using which methodology AGI will actually come into being and yet we have to, we absolutely must align that AGI before it even comes to exist otherwise we might be too late to save the world. Note that here I don't intend to say that how we will attain generalization is the most challenging problem for alignment. I do think that how we will align an AGI is the challenging part, but I think what adds to the challenge is our uncertainty about what the model itself will look like once it gets superintelligence. Again, this is based on the assumption that we don't yet have definitive proof that the AGI will look like GPT-3 or Alphazero, if we do and I am unaware please do let me know of it.

The following image shows a spatial representation of my understanding of existing technical alignment research. This tells the story of how current research is model-dependent and how we are trying to make each model safe separately.

Figure 1: Model-dependent Alignment Research Pathway

My primary cause of concern about existing research is from limitation three here, which says - "Aligning AGI likely involves solving very different problems than aligning today’s AI systems. We expect the transition to be somewhat continuous, but if there are major discontinuities or paradigm shifts, then most lessons learned from aligning models like InstructGPT might not be directly useful." What if the transition is not continuous, w...
The Slippery Slope from DALLE-2 to Deepfake Anarchy
by scasper — 2022-11-05T14:53:54.556Z

OpenAI developed DALLE-2 to be safe. Then StabilityAI made an open-source copycat.

Stephen Casper (

scasper@mit.edu

)

Phillip Christoffersen (

philljkc@mit.edu

)

Rui-Jie Yew (

rjy@mit.edu

)

Thanks to Tan Zhi-Xuan and Dylan Hadfield-Menell for feedback.

A

different version of post

has been posted in the EA forum. It is somewhat longer, focuses more on policy, and is written for a slightly more general audience.

This post talks about NSFW content but does not contain any. All links from this post are SFW.

Abstract

Since OpenAI published their work on

DALLE-2

(an AI system that produces images from text prompts) in April, several copycat text-to-image models have been developed including StabilityAI’s

Stable Diffusion

. Stable Diffusion is open-source and can be easily misused, including for the almost-effortless development of NSFW images of specific people for blackmail or harassment. We argue that OpenAI and StabilityAI’s efforts to avoid misuse have foreseeably failed and that both share responsibility for harms from these models. And even if one is not concerned about issues specific to text-to-image models, this case study raises concerns about how copycatting and open-sourcing could lead to abuses of more dangerous systems in the future. We discuss design principles that developers should abide by when designing advanced AI systems to reduce risks. We conclude that this case study highlights issues with working on risky capabilities and repudiates attempts to achieve AI alignment via racing to advance capabilities.

What’s wrong?

Recent developments in AI image generation have made text-to-image models very effective at producing highly realistic images from captions. For some examples, see the

paper from OpenAI on their DALLE-2 model

or the

release from Stability AI of their Stable Diffusion model

. Deep neural image generators like

StyleGan

and manual image editing tools like Photoshop have been on the scene for years. But today, DALLE-2 and

Stable Diffusion

(which is open source) are uniquely effective at rapidly producing highly-realistic images from open-ended prompts.

There are a number of risks posed by these models, and

OpenAI acknowledges this

. Unlike conventional art and Photoshop, today’s text-to-image models can produce images from open-ended prompts by a user in seconds. Concerns&...