trentmkelly/LessWrong-43k huggingface dataset:
Disproving and partially fixing a fully homomorphic encryption scheme with perfect secrecy
by Lysandre Terrisse — 2024-05-26T14:56:17.233Z

Summary

In the last blog post, I introduced my plan to make the safest cryptographic box in the world, and to make it widely available. This would, in theory, make it possible to run infinitely dangerous programs (including superintelligences) safely. This cryptographic box was supposed to use a scheme that is fully homomorphic and perfectly secret at the same time. However, in the last four months, something has changed:

I discovered that the symmetric Quantum Fully Homomorphic Encryption scheme with perfect secrecy that I was using has a flaw.
I managed to find a partial fix for the scheme and to implement it. I will prove that this fix enables us to safely evaluate every Clifford gate. But this fix still doesn't enable us to safely evaluate every quantum gate.
However, not all hope is lost. Although this specific scheme, made by Min Liang, has a flaw, there is another QFHE scheme with perfect secrecy made by the same researcher. In the future, I will try to implement this second QFHE scheme instead, hoping that it does not have any flaw.

This post explains in more details these points. I will also raise concerns about whether such QFHE scheme could help malevolent actors hide the training of superintelligences. If you know about any other FHE or QFHE scheme with perfect secrecy than the ones I have cited, please let me know.

Prerequisites to understand this post

In quantum computing, quantum gates are represented by matrices, and quantum states are either represented as vectors or as density matrices. If we want to apply a gate G on a state s in the vector paradigm, we compute G⋅s, whereas if we want to do so in the density matrix paradigm, we compute G⋅s⋅G−1. I will use the density matrix paradigm, as it gives more information, and because it is the paradigm used in the article that introduced this QFHE scheme. To understand this post, we need to know about the following quantum gates and quantum states:

X=(0110), Y=(0−ii0), and Z=(100−1) are called the Pauli gates. The most familiar of these is the X gate, which is a generalization of the classical NOT gate. Indeed, it transforms zeros into ones and ones into zeros. The other two gates do not have any analog gate in classical programming. They are called X, Y, and Z because they flip the state of the qubit 180 degrees along the

...
The AI Revolution in Biology
by Roman Leventov — 2024-05-26T09:30:37.997Z
Link: https://www.cognitiverevolution.ai/the-ai-revolution-in-biology-from-vaccines-to-protein-engineering-with-amelie-schreiber/

An expert in the field (Amelie Schreiber) predicts the convergence of AI models for biology to the state where designing drugs, targeted genetic mutations (which then could be enacted through genetic therapy), or viruses for specific purposes becomes really easy in a few years. (Note: apparently, the podcast was recorded before the release of AlphaFold 3.)

Unlike AGI safety, where OpenAI's argument of "iterative deployment" merits more weight[1], in biology I think we can already map the capabilities that will be unlocked. In fact, this podcast does it pretty well, it seems to me. So, it should also be possible to develop effective regulation and controls ahead of the creation of the bio-capabilities.

This raises the questions of the relative speeds of bio-capabilities, regulation, and controls development. And the ethics of pushing bio-capabilities if regulation and controls are lagging behind.

^

Even though many people, including me still don't think "iterative development" is justified the way OpenAI does it.
Who does the artwork for LessWrong?
by Edwin Evans — 2024-05-26T05:55:12.224Z

The art on the LessWrong site and books is amazing. The consistent theme across everything is beautiful!

hm... I just checked the physical book and it credits Midjourney. But I feel there must have been some serious prompt crafting/human artistry behind this?

Is there some human that deserves credit for this?
Is there an idiom for bonding over shared trials/trauma?
by CstineSublime — 2024-05-26T01:18:20.659Z

Is there a idiom or phrase which describes bonding or attachment, not between a victim and an abuser like in Trauma Bonding or Stockholm Syndrome, but between two or more victims: i.e. how would you describe two hostages who forge a friendship or deep connection based on their shared experience? (A common trope of TV show episodes to fast crystalize enemies into friends)
Specifically I'm wondering about a word where they were in the same environment, not sharing similar experiences.
Moloch—An Illustrated Primer
by James Stephen Brown — 2024-05-26T01:04:55.442Z
Link: https://nonzerosum.games/whoismoloch.html

WHO IS MOLOCH?

Last year I came across an interview with an astrophysicist and former professional poker player who read an article about an old poem about an even older god, which may or may not be either an even older mythical creature or potentially an even older word, which represents a particular type of

negative-sum game

, and that particular type of negative-sum game is very relevant right now.

The god in question is the namesake of this here post—Moloch, or Molech, or MLK as the case may be. But before we get to that, welcome to...

THE MOLOCH INTER-DIMENSIONAL SPACEPORT BAGGAGE CLAIM.

For the Thorny Devils arriving at the Moloch Inter-Dimensional Spaceport baggage claim, it takes forever to retrieve their bags, because a few of these prickly characters insist on crowding the conveyor, blocking the vision and the path of those stuck behind them.

Those in front still have to wait for their bag to come to their spot on the conveyor, while those behind have to wait 'til their bag reaches the very end of the conveyor where they might have a chance to push past and retrieve their bag before it vanishes back into the mouth of the beast, some can’t see anything, and those at the front, having retrieved their bags, are trapped by the horde behind them.

For each of these woe-some souls, all they want is to get their bag and get to the taxi stand, but no one is leaving any time soon.

WHO IS TO BLAME?

Moloch.

Moloch is a nasty character. He appears in John Milton’s Paradise Lost as the most bloodthirsty of all the fallen angels and is named in the Bible as a Canaanite god associated with child sacrifice. But there is some confusion over whether Moloch is a god or merely the personification of the Punic word for sacrifice which is “Mlk” or perhaps is a combination of the Hebrew words for king “Melech” combined with the word for shame “bōšet”. There are also many parallels between the bona fides of Moloch and the Ancient Greek Minotaur. Both have a bull’s head on a man’s body and the fury of both are only appeased by ritual sacrifice. So, as with many myths, the origins are hazy.

WHAT DOES MOLOCH MEAN?

Moloch is conceived as a tyrannical god that demands child sacrifice, threatening far worse if his hunger is not satisfied. In game theoretical terms, the negative payoff of losing his support, or worse courting his ire, makes yielding to the tyrant’s demands the optimal ...
Is CDT with precommitment enough?
by martinkunev — 2024-05-25T21:40:11.236Z

Logical decision theory was introduced (in part) to resolve problems such as Parfit's hitchhiker.

I heard an argument that there is no reason to introduce a new decision theory - one can just take causal decision theory and precommit to doing whatever is needed on such problems (e.g. pay the money once in the city).

This seems dubious given that people spent so much time on developing logical decision theory. However, I cannot formulate a counterargument. What is wrong with the claim that CDT with precommitment is the "right" decision theory?
Complex systems theory in human performance. New model for conceptualizing training, adaptation and long-term development
by Matěj Nekoranec — 2024-05-25T20:17:33.986Z

Lately, I’ve been exploring the concept of complex adaptive systems and their relevance to human performance. Every coach or sports scientist understands that we cannot break down performance into isolated parameters and expect that a single parameter will account for the performance as a whole. The function of any complex system depends on the interaction of its components. As Russell Ackoff once said, you cannot decompose a car and assume that the engine alone will take you to your destination.

Every system (we can assume that an athlete is a system) has a primary component known as an attractor, which is a highly stable component of a system and typically does not fluctuate much over time (Balague et al, 2013). Whenever the system is perturbed (due to training or environmental stressors), it seeks an attractor state — a stable position within the system where the system feels most comfortable, in very basic terms.

For instance, an attractor could be something as straightforward as an athlete with kyphosis. The attractor state in which the athlete is situated is typically reflected in their history, training regimen specificity, and length of exposure. It can even extend to emotional and psychological burdens, which may manifest in the athlete’s overall posture. However, for the sake of this discussion, let’s limit ourselves to the physiological and anthropometric domains.

Figure 1: The visualisation shows the attractor state. The attractor landscape is best explained via ‘hilly terrain’ in which we want to move from one state to another. In this particular scenario — the attractor is fixed in the valley and due to the force of gravity it is very hard to escape from. To leave the attractor state, it is necessary to form an adequate landscape, which will facilitate the transition.

When this particular athlete trains using a specific attractor, it is only a matter of time before it begins to affect other physiological systems. For instance, kyphosis can result in inadequate breathing mechanics, suboptimal biomechanical execution, increase risk of injury and so on. Although it is possible to transition from this state to another, doing so requires modifications to the attractor landscape. This change can be brought about by any interventions that specifically target the issue, such as training, rehabilitation, or cultural changes (e.g., joining a new team or adju...
Blindspot in Sport’s Data-Driven Age
by Matěj Nekoranec — 2024-05-25T20:17:33.971Z

In today’s world of sports, we’re seeing a rapid rise in the use of technology, both among amateur enthusiasts and professional teams. This growth involves everything from data acquisition to predictive algorithms. On one side, amateur athletes invest in wearable gadgets that track everything from sleep patterns, and heart rate variability to continuous blood sugar values. Meanwhile, PRO teams are going all-in on advanced prediction models, hiring top-class computer science undergraduates to develop models that can break down the game into an algorithmic language.

As someone who’s fascinated by tech and machine learning, I find it hard to fully jump on the data-driven hype train. Why? Well, most of my career has been formed by coaching, spending enormous amount of time talking to athletes. Anyone coming from the coaching perspective understands that first-person athlete experience is a key feature of any successful development of an athlete. Trying to boil all of that down to a discrete data point just doesn’t sit right with me. In this discussion, I’ll explain why I think the current explosion of data in sports won’t lead to never-ending exponential growth but rather to a sigmoid curve. And I’ll also talk about why that might actually be a good thing for the future of sports.

Human vs AI — model specification

To begin the discussion, let’s start with a brief crash course on neuroscience. Instead of observing ourselves from a third-person perspective (what science usually does), let’s consider what it means to be in the eyes of the observed person, meaning first-person perspective. One of the fundamental forces that drives our decision-making is known as predictive coding. Our brains create probabilistic maps between our actions and their outcomes based on input from sensory organs such as vision or proprioception. When an action reduces uncertainty, it triggers a positive reinforcement loop for that behaviour; when predictions are far off, usually defined as a prediction error, it prompts either belief adjustment or a selection of actions aligned with top-down predictions (Den Ouden et al., 2012). In simple terms, we select actions that correspond to our beliefs.

Let’s now shift our focus to examine the current state-of-the-art large language models. Chief AI architect at Meta Yan Le Cunn provided an example of why today’s language models fall far short of huma...
LMSR subsidy parameter is the price of information
by Abhimanyu Pallavi Sudhir — 2024-05-25T18:05:31.998Z

A logarithmic scoring rule to elicit a probability distribution r on a random variable X∈{1…n} is s(r)=blog(rX). Something that always seemed clear to me but I haven’t seen explicitly written anywhere is that the parameter b is just the price of information on X.

Firstly: for an agent with true belief p, the expected score from making a report r is Ep[s(r)]=∑x∈{1…n}blog(rx)px=−bH(p,r) where H is cross-entropy. This is maximized when r=p.

Well, this is just the standard proof that logarithmic scoring is proper. This max score itself is Ep[s(p)]=−bH(p) i.e. the entropy in p. So your expected earning is exactly proportional to the information you have on X (the negative of the entropy in your probability distribution for it), and the proportionality constant, the price of a bit of information on X, is b.

This can be made even clearer by considering the value of some other piece of information Y. If Y=y and you learn this fact, you will bet P(X|Y=y) which would give you an expected score of EP(X∣Y=y)[P(X∣Y=y)]=−bH(P(X∣Y=y)). Taking the expectation over Y, your expected score if you acquire Y is −bEP(Y)[H(P(X∣Y=y))] which is the conditional entropy −bH(X∣Y). Thus the expected profit from acquiring Y is −b(H(X∣Y)−H(X))=bI(X;Y).

So the value of Y is precisely b multiplied by its mutual information with X, i.e. b is the price of one bit of information on X.

I assume this is widely known. But I think it’s still pedagogically useful to actually think in these terms because it sheds light on things like:

Choosing a good scoring rule — it’s not just that we want the scoring rule to be proper (incentivize honesty), we also want it to incentivize the optimal amount of effort in acquiring information. b should be a measure of how important a question is.

But also: like the price of any good, the price of information can vary (e.g. you might want to reduce b after some key information becomes public, since you’re getting it for free)! And like any good it would have diminishing returns. This motivates things like [1].

It makes clear the fact that prediction markets have huge positive externalities — the market-maker is paying for the information, but it becomes public. This is bad (see also: [2]) — in general, IP rights remain an unsolved problem: [3]. I have a very clever idea to solve it, which I will elaborate in another post.

[1] “Market Making with Decreasing Utility for ...
Low Fertility is a Degrowth Paradise
by Maxwell Tabarrok — 2024-05-25T17:35:23.627Z
Link: https://www.maximum-progress.com/p/low-fertility-is-a-degrowthers-paradise

“Actually, the problem in the world is that there are too many rich people.”

Paul Ehrlich

Degrowth is an ascendant cultural and political movement. Its central claim is that the growth of humanity’s population and economy is unsustainable on a planet with finite resources. Therefore, the only way to avoid inevitable future collapse and incalculable damage to the earth’s natural environment is to voluntarily slow and reverse this growth now. The cultural values and policy prescriptions of degrowth are shared by prominent

political activists

,

scientists

, and

world leaders

.

I’ve written

several

posts

on

this blog about low and falling global fertility. I’ve always framed low fertility as a big problem, as indeed it is from a Progress Studies perspective. If you care about continuing the economic growth and technological progress that has created the modern world, low fertility is a massive challenge.

From a degrowth perspective, low fertility is a blessing. Paul Ehrlich helped India

forcibly sterilize

millions of women. A massive human rights violation, but one he saw as justified given the grave dangers he foresaw with high population growth rates. Today,

India’s fertility rate

is below replacement for completely voluntary cultural and economic reasons and the global average isn’t far behind. Within the framework of degrowth, sub-replacement fertility and the shrinking economies that come with it aren’t problems to be solved, they are the necessary adaptations to global environmental limits.

The Empty Planet Result

Consider how these different perspectives would react to this paper by Chad Jones:

The End of Economic Growth? Unintended Consequences of a Declining Population

.

A total fertility rate slightly above 2 and one slightly below 2 is the difference between an exponentially growing population and an exponentially declining one. In his paper Jones shows that when you plug in exponentially declining population into the standard models of economic growth you get the Empty Planet Result:

Economic growth stagnates as the stock of knowledge and living standards asymptote to constant values. Meanwhile, the population itself falls at a constant rate, gradually emptying the planet of people. This outcome stands in stark contrast to the conventional result in growth models in which knowledge, living standards, and even population grow exponentially: not only do we g

...
Episode: Austin vs Linch on OpenAI
by Austin Chen — 2024-05-25T16:15:38.790Z
Link: https://manifund.substack.com/p/episode-austin-vs-linch-on-openai

Austin (of Manifund) and Linch (of EA Funds) explore our disagreements on OpenAI. Austin is approximately an apologist for OpenAI, and for-profit/tech companies overall; while Linch takes a more hardline stance.

Neither of us are experts on OpenAI or have insider information; we’re just hashing out our different worldviews in a casual setting. If you listen or follow along, let us know what you think!

Topics covered:
00:33 OpenAI's contradictions
02:44 NDAs and tech vs rationalist culture
13:51 Is there enough criticism of OpenAI? Too much?
17:15 Superalignment's 20% compute promise
21:50 The EA departures from OpenAI
26:10 ScarJo
34:41 Should AGI be pursued by a for-profit company?
46:10 A hypothetical AGI "Manhattan Project"
51:45 Government vs company services
54:08 Is OpenAI democratic?

Note that this podcast was recorded May 22, before Kelsey Piper’s expose on NDAs; subsequent NDA walkbacks by OpenAI; and reports that Sky was indeed commissioned before the ScarJo reachout

Link with video & transcripts:

https://share.descript.com/view/1N6kvfXFLr9

1: OpenAI

Austin: [00:00:00] So we're here today to talk about OpenAI a subject that Linch and I have talked about in some context before.

I think I, coming from a more tech startup background, have views that maybe are a little bit more alien to the OpenAI. What I think of as a standard EA like AI safety, maybe Doomer ish views, which maybe Lynch represents a little bit more of.

So I'm excited to have a chance to hash this out with you today, Lynch.

Linch: Yeah, happy to do I think if anything, I'm probably more negative about like OpenAI than the average like EA person, like for example, the OpenPhil cluster viewpoints. Interesting.

[00:00:33] OpenAsteroidImpact & OpenAI's contradictions

Austin: And when you created Open Asteroid Impact, right?

This is like a satirical thing, trying to get people to pay more attention to OpenAI and like ways that it could go. Badly. And I was working with you on that. I think in my mind, I didn't even think of this as " Oh, this is because we want to draw attention to like things, OpenAI is doing badly."

Rather like it just seems funny to me.

Linch: Yeah. Yeah. I think badly is like one conception of it. I think contradiction is another conception. Like I was trying to highlight the ways in which I thought the public messaging, or at least Yeah. Public messaging as understood by p...
Training-time domain authorization could be helpful for safety
by domenicrosati — 2024-05-25T15:10:45.013Z

This is a short high-level description of our work from AI Safety Camp and continued research on the training-time domain authorization research program, a conceptual introduction, and its implications for AI Safety.

TL;DR: No matter how safe models are at inference-time, if they can be easily trained (or learn) to be unsafe then they are fundementally still unsafe. We have some ways of potentially mitigating this.

Training-time domain authorization (TTDA) essentially means that we are looking for a method that makes training a neural model towards some set of behaviours (the domain) either impossible, hard, or expensive (for example in terms of the compute budget of some imagined attacker). Another framing that might fit better in an RL setting is that we are looking for a method that makes learning a specified policy from some feedback impossible, hard, or expensive. This is in contrast to inference-time domain authorization: methods that prevent a neural model from behaving certain ways. Much of mainstream value-alignment of neural w.r.t human values such as RLHF and conventional safety guards are largely concerned with inference-time domain authorization. This distinction may seem artificial but it helps us draw a conceptual line that we will see allows focused distinct technical questions that for better or worse allow us to ignore inference-time domain authorization (or take it for granted).

The fundemental motivating argument for TTDA sates that no matter how safe models are at inference-time, if they can be easily trained to be unsafe then they are fundementally still unsafe [Motivating Argument of TTDA]. (The opposite is true as well which is why inference-time domain authorization is an equally important line of research to continue). The motivating argument of TTDA is especially concerning in a world where we continue to release model weights in the open but even without open release: weight stealing and fine-tuning APIs make TTDA an important topic of consideration.

TTDA is not a general solution to safety or alignment (infact we will need to specific which domains to authorization which is a fundemental alignment problem) but we argue it is a critical piece of the puzzle especially for so-called "near term" risks such as the assistance with weapons development, massive scale illegal content generation or fraud campaigns, etc.

We plan on putting...