trentmkelly/LessWrong-43k huggingface dataset:
Easy fixing Voting
by Charbel-Raphaël — 2022-10-02T17:03:20.566Z

[I forgot to post this post during the elections, but still relevant]

Context : Two round uninominal ballot

At this moment in France, it's the election period. And between two existential anguishes about AGIs, I sometimes think about the presidential campaigns.

We use a two round uninominal ballot: in each round, each citizen votes for a single candidate.

Problem : Bad properties

The uninominal voting system really does suck because primaries are not mandatory, so some candidates who could ally themselves do not ally themselves. So you have to vote strategically, and that has a lot of bad properties.

A classic example that highlights the bad properties of the uninominal ballot is the case where the majority of the population is left-wing, but 10 left-wing candidates are running while only one right-wing candidate is running. It is then the right wing that wins the election because of the spoiler effect, i.e. the dispersal of the votes of the left wing candidates.

Another problem is that strategic voting prevents conviction voting. It is only interesting to vote for those who have a chance to win.

Easy fix : the vote by approval.

There are a lot of possible voting systems (ranking, randomized condorcet, bordas point ballot, majority judgment ballot...). Which one to choose? Condorcet voting is relatively popular on this platform, but the main problem is its difficulty to implement and its complexity to vulgarize. The voting systems with the best properties are also unfortunately difficult to use.

The French YouTuber Mr Phi proposes a more pragmatic goal: By changing the minimum of rules possible, how to improve the elections as much as possible? What would be a credible and feasible alternative in the near future?

Pourquoi notre système de vote est nul (et le moyen le plus simple de l'améliorer)

[Why our voting system is bad]

The answer presented is very simple: instead of voting for a single person, you can endorse one or several candidates. In practice, it would be enough to remove the constraint of a single name in the envelope, giving the possibility to vote for several candidates at the same time in the envelope.

This solution is straightforward because we would keep all the rest of the election organization: we count the names, and who has the most wins. We could even take the opportunity to eliminate the second round.

Link with effective altruism?

In theory, app...
Monthly Shorts 9/22, and An Essay in Defense of Technodeterminism
by Celer — 2022-10-02T16:10:09.512Z

Possibly all our fear over declining interstate migration is a …. data artifact? h/t Evan Soltas.

CPS data seem to noticeably diverge from the rest of the models, which I think it’s fair to say is suspicious.

So, you know how everything started going wrong in America between 1969 and 1980? That’s also when the age of senators started rising.

You can pay money to get taller! 3-6 inches, and months of extremely painful recovery. Price tag, $70-$150k. We’re getting to the glorious transhumanist future day by day

The Canvas Cycle is a series of pretty images. All done in exactly 256 color options, and instead of rotating an actual image, individual locations get changed to different colors at a set speed. Look at the options and play around: it’s a really fun art style.

One of my favorites

So, the reason I haven’t been reading much this month is that I’ve been writing. A lot. Some of it you will be able to see eventually (I hope). And I’ve been going back to a piece by Anton Howes I read early in September on the balance of writing, something that I think is more true than many people want to admit. It makes one key point:

If you’re stumped writing, it’s probably because you’re not done thinking.

Finally, I leave you with a short piece I’ve written, because it’s been percolating for a while. I’d love feedback if you want to argue with it.

In Defense of Technodeterminism

The technodeterminism thesis:

The state of technology is the most important cause of social structures, because technology enables most human action. People act in the context of available technology, and therefore people’s relations among themselves can only be understood in the context of technology.

From An Introduction to Science and Technology Studies, by Sergio Sismondo. You can argue about the details, but it’s a description by someone who is largely critical, and I like it as someone who is largely positive: it will do.

It can seem a little too strong: what about love? The technodeterminist can respond that trying to understand any type of love in a subculture where men spend six months of the year away on boats in homosocial environments without taking that fact into account is futile, and if other societies need not consider that aspect it is because of differing technologies. Similarly, the ability of people to write letters to each other, or make video-calls, structures our artistic interactions,...
Open & Welcome Thread - Oct 2022
by niplav — 2022-10-02T11:04:06.762Z

If it’s worth saying, but not worth its own post, here's a place to put it.

If you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are invited. This is also the place to discuss feature requests and other ideas you have for the site, if you don't want to write a full top-level post.

If you're new to the community, you can start reading the Highlights from the Sequences, a collection of posts about the core ideas of LessWrong.

If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ. If you want to orient to the content on the site, you can also check out the Concepts section.

The Open Thread tag is here. The Open Thread sequence is here.
Any further work on AI Safety Success Stories?
by Krieger — 2022-10-02T09:53:45.309Z

TL;DR, Are they any works similar to Wei_Dai's Ai Safety "Success Stories" that provide a framework to think about the landscape of possible success stories & pathways humanity will take to survive misaligned AI?

I've been trying to think of systematic ways of assessing non-technical proposals for improving the odds of humanity's survival from misaligned AI.

Aside from numerous frameworks for assessing technical alignment proposals, I haven't seen much resources on non-technical proposals that provide a concrete framework to think about the question: "What technological/geopolitical/societal pathway will our civilization most likely take (or should ideally take) in order to survive AI?"

Having such a framework seems pretty valuable since it would let us think about the exact alignment-pathway & context in which [proposals that want to help with alignment] would be effective at.

For example, a pretty clear dimension in which people's opinions differ is in the necessity of pivotal acts i.e. "pivotal act vs gradual steering" (kind of oversimplified)—here, any proposal's theory of impact will necessarily depend on their beliefs regarding (a) which position on the spectrum currently appears to be most likely by default, and (b) which position on the spectrum we should be aiming for.

If, say, my pessimistic containment strategy was about communicating AI risk to the capabilities people in order to promote cooperation between AI labs it would be incoherent for me to, at the same time, be ultra-pessimistic about humanity's chances of enacting any cooperative regulation in the future.
Or if I thought a Pivotal Act was the best option humanity has, and wants to suggest some proposal that would be a force-multiplier if that line of strategy does happen in the future, it would make sense for my proposal to consider the forms in which this unilateralist org's AI will take.

where will it developed?
will it be a corrigible AI whose safety features depend on human operators?
will it be a CEV-type AI whose safety feature won't depend on humans?
how likely is it that the first AI capable of enacting a Pivotal Act will need to rely on human infrastructure—for how long—and could interventions help?

I've seen a lot of similar frameworks for technical alignment proposals, but nothing much for pathways our civilization will actually take to survive (Wei_Dai's post is similar, but is ...
Request for feedback on sample blurbs for the EA fantasy novel I wrote
by Timothy Underwood — 2022-10-02T09:01:58.678Z

Thanks everyone for the help and comments, I’ve taken everything I read seriously, and changed a lot of how the main character thinks about EA topics due to it — I am now looking for more readings that are specifically aimed at finding stuff that might accidentally offend people before I send it out into the world. So if anyone knows a non-ea friend who might enjoy reading something like this, and who is willing to leave comments about what bothered them, I would find that really useful.

Otherwise, my current plan is that in a few weeks I’ll start publishing it through Royal Road, Spacebattles forum, my own website and Amazon (though I’m undecided between the 2.99 price point or the .99c price point) for publication.

I am still actively looking for other places to publish it, and if anyone can help with promoting this book, that would be useful.

Also if you haven’t yet read it, but do want to comment and tell me what definitely needs to be changed, here is the

shared google doc.

Blurbs

Also, here are three possible versions of the blurb that I’ll be using when I publish it. What do you all think about them? I think it is important to include in the blurb that I received some support while writing this novel.

===

After Isaac died and woke up as a powerful cultivator in a fantasy world, he wanted to do as much good as he could possibly manage.

But it turned out to be hard to figure out what actually was the single most useful thing to do.

And the guy whose body he now occupied, wasn’t actually dead, and he had his own goals that weren’t the same as Isaac’s…

Note: While it is supposed to be a fun story, a major goal of this novel is to describe the ideas of Effective Altruism, and I received a grant from an EA associated organization to help write it.

===

I’d died in a car crash, and woke up as a powerful cultivator in a new world.

Honestly, this was pretty awesome, even though there was a war coming to the island that I’d woken up on, which I wanted to have nothing to do with. I wanted to help as many people as I could, as much as I could, and dying in a war that I had nothing to do with wouldn’t help anyone.

I really didn’t want to die again.

But it turned out that the cultivator whose body I’d found myself in wasn’t dead, and he was absolutely determined to fight and probably die.

Note: This novel was partly written to promote th...
Paper: Large Language Models Can Self-improve [Linkpost]
by Evan R. Murphy — 2022-10-02T01:29:00.181Z
Link: https://openreview.net/forum?id=NiEtU7blzN

This paper came out about a week ago. I am not the author - it was published anonymously.

I learned about the paper when JJ Hepburn shared it on Slack. I just thought it seemed potentially really important and I hadn't seen it discussed on this forum yet:

Paper title: "Large Language Models Can Self-improve"

Author: Anonymous

Abstract: "Large Language Models (LLMs) have achieved excellent performances in various tasks. However, fine-tuning an LLM requires extensive supervision. Human, on the other hand, may improve their reasoning abilities by self-thinking without external inputs. In this work, we demonstrate that an LLM is also capable of self-improving with only unlabeled datasets. We use a pre-trained LLM to generate “high-confidence” rationale-augmented answers for unlabeled questions using Chain-of-Thought prompting and self-consistency, and fine-tune the LLM using those self-generated solutions as target outputs. We show that our approach improves the general reasoning ability of a 540B-parameter LLM (74.4%→82.1% on GSM8K, 78.2%→83.0% on DROP, 90.0%→94.4% on OpenBookQA, and 63.4%→67.9% on ANLI-A3) and achieves state-of-the-art-level performance, without any ground truth label. We conduct ablation studies and show that finetuning on reasoning is critical for self-improvement."

Link: https://openreview.net/forum?id=NiEtU7blzN
Four usages of "loss" in AI
by TurnTrout — 2022-10-02T00:52:35.959Z

Summary: What does it mean for a loss function to be "aligned with" human goals? I perceive four different concepts which involve "loss function" in importantly different ways:

Physical-loss: The physical implementation of a loss function and the loss computations,
Mathematical-loss: The mathematical idealization of a loss function,
A loss function "encoding/representing/aligning with" an intended goal, and
Agents which "care about achieving low loss."

I advocate retaining physical- and mathematical-loss. I advocate dropping 3 in favor of talking directly about desired AI cognition and how the loss function entrains that cognition. I advocate disambiguating 4, because it can refer to a range of physically grounded preferences about loss (e.g. low value at the loss register versus making perfect future predictions).

Related: Towards deconfusing wireheading and reward maximization.[1] I'm going to talk about "loss" instead of "reward", but the lessons apply to both.

I think it's important to maintain a sharp distinction between the following four concepts.

1: Physically implemented loss

The loss function updated my network.

This is a statement about computations embedded in physical reality. This statement involves the physically implemented sequence of loss computations which stream in throughout training. For example, the computations engendered by loss_fn = torch.nn.CrossEntropyLoss().

2: Mathematical loss

The loss function is a smooth function of the prediction distribution.

This is a statement about the idealized mathematical loss function. These are the mathematical objects you can prove learning theory results about. The Platonic idealization of the learning problem and the mathematical output-grading rule casts a shadow into your computer via its real-world implementation (concept 1).

For example, (D,ℓ) where D:={(x,label(x))∣x∈MNIST} is the mathematical idealization of the MNIST dataset, where the x∈R28×28 are the idealized grayscale MNIST images. And ℓ is the mathematical function of cross-entropy (CE) loss between a label prediction distribution and the ground-truth labels.

3: Loss functions "representing" goals

I want a loss function which is aligned with the goal of "write good novels."

This is an aspirational statement about achieving some kind of correspondence between the loss function a...
The computational complexity of progress
by Malmesbury — 2022-10-01T21:50:55.709Z

Cross-posted from Telescopic Turnip.

Artem Kaznatcheev has a curious model about the evolution of living things. When a population of creatures adapts to their environment, they are solving a complicated mathematical problem: they are searching for the one genome that replicates itself the most efficiently, among a pantagruelian pile of possible sequences. The cycles of mutation and natural selection can be seen as an algorithm to solve this problem. So, we can ask, are some evolutionary problems harder than others? According to Kaznatcheev, yes – some features are easy to evolve, some others are outright impossible, such that they can never be reached on planetary timescales. To tell the difference, he defines what he calls the computational complexity of a fitness landscape.

Computational complexity is a term from computer science. It describes how many steps you need to solve a given problem, depending on the size of the problem. For example, how long does it take to sort a list of n items, as a function of n? (If you do it right, the number of operations for sorting is proportional to n×log(n), written O(n log n). Unless you happen to be sorting spaghetti, then you can do it in n operations.) What is the equivalent of this for biological evolution?

Mountains and labyrinths

To get an intuition, consider a bug that can only mutate on two dimensions: the size of its body and the length of its legs. Each combination of the two has a given fitness (let's define it as the number of copies of its genome that gets passed on to the next generation). So we can draw a 2D fitness landscape, with red being bad and cyan being better:

In the middle, there is a sweet spot of body mass and leg length that works great. Say our bug starts in the bottom left; after each generation it will generate a bunch of mutant offspring which spread around the current position. Due to natural selection, the mutants closest to the center will reproduce more, and if you repeat the process, the bug population will gradually climb the gradient towards the top of the fitness mountain.

Evolution in O(eⁿ)

The mountain range metaphor is actually pretty bad. Kaznatcheev offers a better one: a fitness landscape is a maze. Let's look at two other examples:

Like the first one, these have no local maxima, there is only one global fitness maximum and, as long as you climb the gradient, you should even...
Company leadership and tactical decisions
by Dentin — 2022-10-01T21:21:39.269Z

The other day at work, I received an email from our leadership regarding "new principles for our organization", linking out to a web page listing the new principles. This ended up being partially a culture document, but mostly a direction document - "we want customers to be able to X" "we want to provide industry standard Y", etc. While the wish list was pretty explicit and fairly detailed, there were no concrete actions.

At the level that our leadership works at, I believe this is largely the correct thing to do. If you're trying to set direction for a thousand people, you can't really specify technical details, and even trying to promote specific strategies is almost always going to end up with suboptimal results. Plus, if you hire smart, motivated people, they'll find good (or good enough) solutions.

Except in the case of a systemic defect or dysfunction which keeps most or all of your teams trapped in local minima, while there's a better global minimum available.

I believe this happens in a lot of companies, and I believe it's happening in mine. The new principles document sets goals that implicitly require a lot of stabilization and polishing of our products - we're trying to support customer use cases, but our development so far has been a bleeding edge iterate-as-fast-as-possible mess.

In an ideal world, the smart people in my organization would recognize this, and globally collaborate to focus on the stabilization work that's needed. However, in the real world, my organization has significant social and financial incentives around creating new features and adding functionality, along with an embedded culture of promoting those things. There are almost no incentives around stabilization and polishing of products. This results in only new features and functionality being prioritized. Everything else gets the bare minimum of effort needed to keep our services up, keep things going.

It's a systemic problem across the entire organization, and it's one that can't really be solved at the local level, because of the cultural and financial issues.

This is where our leadership can, and perhaps should step in. Sometimes, direction and veiled hints just aren't enough. Sometimes, as a leader, you might want or need to make a hard, concrete tactical decision that actually has consequences if you get it wrong, and use...
Clarifying Your Principles
by Raemon — 2022-10-01T21:20:33.474Z

Following up on:

Integrity and Accountability are Core Parts of Rationality
Robust Agency for People and Organizations

It's easy to end up in situations where people don't know which principles you stand for. (And, which ones you'll really stand for when it's inconvenient).

It's easy to end up in situations where you don't know what principles you stand for, and which ones you'll actually stand up for when it's inconvenient.

You can get away without knowing exactly what your principles are, for awhile. But if you want others to trust you in high stakes situations, it's helpful if you've proactively made your principles clear, and demonstrated that you can live by them. Most people don't really stand up for principles when it's inconvenient, so your prior should be that you probably won't, either.

Being clear about your principles requires you to have them in the first place, and to practice the muscle of standing up for them, so that you know they are real and not just a vague applause light you are professing.

Integrity Debt

In software development, sometimes you need to write messy code that will cause problems for you later on down the line. Writing "good code" would take too long, and you need to ship your product. But, eventually, this messiness is going to make it harder for you to make progress on the codebase, and you'll want to spend some time simplifying your code to "pay down the debt."

Similarly, if you're starting a new organization that depends on trust (either with the public, or particular stakeholders), there's a bunch of actions you might take to build that trust... which you may not have time to do when you're getting started.

Integrity debt accumulates most acutely when you make controversial judgment calls that are hard to explain. Or, judgments that you reflectively wouldn't endorse. I think it also accumulates in small ways, when you do non-controversial but nonetheless confusing things.

But perhaps more importantly (if subtly), integrity debt accumulates when you take on responsibilities that require you to have principles, without yet knowing what those principles are. This may work initially, but eventually will be like building a castle on a foundation of sand. Sooner or later you need to figure out the principles underlying you, your project, or your organization.

If you've undertaken responsibilities without understan...