trentmkelly/LessWrong-43k huggingface dataset:
Ceiling Air Purifier
by jefftk — 2022-05-30T19:20:02.130Z

Imagine a high-capacity air filter that is extremely quiet and doesn't
get in the way. I think I see how to build one; here's a prototype:

By putting a fan on the ceiling you can have very long blades. This
is important for keeping noise down: the longer your blades are the
more air you move at a given rotational velocity, which is a major
determinant of noise. Testing my prototype, it has a CADR of ~180 CFM
and is only 33dB. By contrast, the Wirecutter's top-recommended air
purifier (Coway
AP-1512HH) has a CADR of 233 CFM (31% higher) at 54 dB (21dB
louder) or 110 CFM (38% lower) at 36 dB (3dB louder). With some
tweaks it should be able to match the commercial purifier's
performance, without being louder.

Volume is very important here: people often select purifiers based
on their maximum flow, but then run them on a lower setting or even
turn them off because they are too loud.

Note that while my purifier is an octagon of 12x20 filters
around a 52" fan, a nonagon would have been better. Not only should
that get you 12% more filtration, but you don't have to fill gaps
between the filters.

Jeff for scale

One downside of using this much tape, especially some of it with
the sticky side bare, is that you get an adhesive smell. The smell
fell below where I could detect after about a day, but Julia could
still smell it for about five days.

To determine the CADR of the prototype I followed the approach I
described in Testing Air
Purifiers, where I created smoke by burning matches and then
tracked how quickly the purifier could reduce pm2.5 levels:

To determine volume levels, I used a Sennheiser 835s dynamic
microphone with a foam windscreen, connected to an AudioBox 1818VSL to
record samples of each purifier option and silence. Here are the
samples:

(Silence)

(Ceiling Purifier)

(AP-1512HH Medium)

(AP-1512HH High)

To compute the loudness of each sample I ran:

ffmpeg -i sample.wav -af\
loudnorm=I=-16:dual_mono=true:TP=-1.5:LRA=11:print_format=summary\
-f null -

This gives loudness in LUFS, which is K-weighted dB. I don't have any
calibrated tooling, but I can report values in loudness units (LU)
relative to silence. Trusting the manufacturer's rating of 53.8 dB
for the AP-1512HH on High, however, lets us determine absolute
loudness for the other options:

Sample
Relative

...
Notion template for personal predictions
by Arjun Yadav — 2022-05-30T17:47:12.650Z

I created a simple Notion template for creating personal predictions. You can set multiple reminders and even embed line charts showcasing the likelihood across time (though this does rely on an external service).

This was created pretty quickly, and the predictions in the template are just for demonstration purposes. I would love to get feedback/suggestions!
Six Dimensions of Operational Adequacy in AGI Projects
by Eliezer Yudkowsky — 2022-05-30T17:00:30.833Z

Editor's note: The following is a lightly edited copy of a document written by Eliezer Yudkowsky in November 2017. Since this is a snapshot of Eliezer’s thinking at a specific time, we’ve sprinkled reminders throughout that this is from 2017.

A background note:

It’s often the case that people are slow to abandon obsolete playbooks in response to a novel challenge. And AGI is certainly a very novel challenge.

Italian general Luigi Cadorna offers a memorable historical example. In the Isonzo Offensive of World War I, Cadorna lost hundreds of thousands of men in futile frontal assaults against enemy trenches defended by barbed wire and machine guns. As morale plummeted and desertions became epidemic, Cadorna began executing his own soldiers en masse, in an attempt to cure the rest of their “cowardice.” The offensive continued for 2.5 years.

Cadorna made many mistakes, but foremost among them was his refusal to recognize that this war was fundamentally unlike those that had come before. Modern weaponry had forced a paradigm shift, and Cadorna’s instincts were not merely miscalibrated—they were systematically broken. No number of small, incremental updates within his obsolete framework would be sufficient to meet the new challenge.

Other examples of this type of mistake include the initial response of the record industry to iTunes and streaming; or, more seriously, the response of most Western governments to COVID-19.

As usual, the real challenge of reference class forecasting is figuring out which reference class the thing you’re trying to model belongs to.

For most problems, rethinking your approach from the ground up is wasteful and unnecessary, because most problems have a similar causal structure to a large number of past cases. When the problem isn’t commensurate with existing strategies, as in the case of AGI, you need a new playbook.

I've sometimes been known to complain, or in a polite way scream in utter terror, that "there is no good guy group in AGI", i.e., if a researcher on this Earth currently wishes to contribute to the common good, there are literally zero projects they can join and no project close to being joinable. In its present version, this document is an informal response to an AI researcher who asked me to list out the qualities of such a "good project".

In summary, a "good project" needs:

Trustworthy comma

...
My SERI MATS Application
by Daniel Paleka — 2022-05-30T02:04:06.763Z

Thanks to Flo Dorner for feedback on the technical content in this post.

I have recently been accepted to the SERI ML Alignment Theory Scholars Program program, where applicants are paired with mentors. The application process was nonstandard: each mentor has a set of open-ended questions on safety research you need to answer in short essay form.
I am posting my application below. Reasons:

It might help future applicants to similar programs overcome writing anxiety;
I want to share the ideas from my answers.

I decided against editing my application post-hoc, to preserve a real "example application" for reference. There were some minor writing corrections. [1] Note: the answers were written in a single pass in a short amount of time.

Alex Gray: Language model risks

Pick a risk that might be present in systems that use very large language models in the future. (...) What risk did you pick? Describe it in your own terms.

It is likely that in future language models, interpretability will be one of the main methods to (try to) enforce safety. One simple interpretability method is making the multi-layer model X "articulate its thoughts" at each layer, by returning a natural language description of the representation at each layer. Think logit lens, but with longer outputs. In what follows, we call the resulting outputs "thought descriptions".

To keep the model safe, one could do human inspection of the thought descriptions, or more scalably, automated detection of "harmful" thought descriptions through a model Y. We could train Y on a large dataset of harmful texts and AI apocalypse scenarios.

Why would this even work, given that now we need another large model, which is trained on very harmful data? Because, the harmful thought detector Y does not need to be as capable as the model X to detect harmful thoughts well. The model Y wins when it detects any harm probability, while X has to work perfectly each time.

We do not need to use any of the above to train X; the described method would best serve as a detection method for safety risks of models trained in other ways.

There are two risks with this method:

The model X learns to fool the "thought articulation method", making its internal representation always be translated into inherently non-harmful thought descriptions.
The first risk is solved in some way, and we can rely on thought descripti

...
Reshaping the AI Industry
by Thane Ruthenis — 2022-05-29T22:54:31.582Z

The wider AI research community is an almost-optimal engine of apocalypse. The primary metric of a paper's success is how much it improves capabilities along concrete metrics, publish-or-perish dynamics supercharge that, the safety side of things is neglected to the tune of 1:49 rate of safety to other research, and most results are made public so as to give everyone else in the world a fair shot at ending it too.

It doesn't have to be this way. The overwhelming majority of the people involved do not actually want to end the world. There must exist an equilibrium in which their intentions match their actions.

Even fractionally shifting the status quo towards that equilibrium would have massive pay-offs, as far as timelines are concerned. Fully overturning it may well constitute a sufficient condition for humanity's survival. Yet I've seen precious little work done in this area, compared to the technical questions of AI alignment. It seems to be picking up in recent months, though — and I'm happy to contribute.

This post is an attempt at a comprehensive high-level overview of the tactical and strategic options available to us.

1. Rationale

Why is it important? Why is it crucial?

First. We, uh, need to make sure that if we figure alignment out people actually implement it. Like, imagine that tomorrow someone comes up with a clever hack that robustly solves the alignment problem... but it increases the compute necessary to train any given ML model by 10%, or it's a bit tricky to implement, or something. Does the wider AI community universally adopt that solution? Or do they ignore it? Or do the industry leaders, after we extensively campaign, pinky-swear to use it the moment they start training models they feel might actually pose a threat, then predictably and fatally misjudge that?

In other words: When the time comes, we'll need to convince people that safety is important enough to fuss around a bit for its sake. But if we can't convince them to do that now, what would change then?

I suppose having a concrete solution, instead of vague prophecies of doom, would give us more credibility. But... would it, really? And what if we won't have a concrete solution even then, just a bunch of weird heuristics that may nonetheless measurably improve our odds?

The latter seems reasonably likely, too. As this excellent post points out, most of the contemporary deep-learning progre...
The Unbearable Lightness of Web Vulnerabilities
by aiiixiii — 2022-05-29T21:13:17.103Z
Link: https://www.theoreticalstructures.io/2022/05/27/the-unbearable-lightness-of-web-vulnerabilities/

We should seriously get better at Cybersecurity.
I discovered a zero day vulnerability affecting thousands of Internet exposed video surveillance devices all around the world.
Finding the Right Problem
by tobot — 2022-05-29T17:52:56.791Z

Modified Hamming Question

There are a few posts in Lesswrong about the Hamming Question

What is the most important problem in your field?

I'd like to ask a modified question:

What is the most important problem in any field for which you specifically have a comparative advantage in solving?

This post is a collection of strategies for finding the answer to the first half of this question. Determining your own comparative advantage to solving these problems is left as an exercise to the reader.

You're Special

Readers of Lesswrong may come to a consensus on the most important existential threats to the human race. Advocates of Effective Altruism may come to a consensus on the most effective ways to save lives. However, there can be no consensus on the modified Hamming question.

If you're a Lawyer, Pharmacists, CEO, or Biologist, your vocation probably has not honed specific skills that give you a comparative advantage in solving the AI alignment problem. The best you might be able to contribute on this problem would be strategic donation of money, however, even in this activity, you possess no comparative advantage. Donation of your money to this problem may just be more effective than the donation of your mental or physical efforts.

However, if you can find an appropriate problem, perhaps one in which no one else with your particular talents and knowledge is currently working to solve, you might be able to contribute considerably more to the welfare of the human race than by the donation of your money alone.

The answer to the modified Hamming question will inevitably be specific to each person (or at least, specific to groups with a large overlap in your skills and knowledge), and since in every field, we see diminished marginal utility from each additional researcher, the answer will also inevitably be one which is not already identified by those in your field.

Search Strategies

With the question and motivation outlined above, below are some problem search strategies.

Survey Everything

You can systematically search all categories for inspiration.

Wordnet provides a convenient way to peruse semantic relationships. You can also access wordnet programmatically for more complex searches.

Similarly, you can peruse Wikidata both manually and programmatically.

Economic Analysis

One way to estimate the impact of a solution is to consider to total economic activity that it...
The impact you might have working on AI safety
by Fabien Roger — 2022-05-29T16:31:48.005Z

This is an attempt at building a rough guess of the impact you might expect having with your career in AI safety.

Your impact will depend on the choices you make, and the estimation of their consequences also depends on your beliefs about AI safety, so I built an online tool so that you can input your own parameters into the model.

The main steps of the estimation

Compute when a world-ending AGI might happen, if there has been no AI safety progress,
Compute when the AI alignment problem might be solved,
Compute your impacts on timelines, and see how it affects the odds that humanity survives.

Main simplifying assumptions

Unsafe AGI being built and AI alignment being solved are independent events: progress towards AGI doesn’t speed up AI safety, but AI Alignment prevents building an unsafe AGI.
AI alignment is either solved or not: if it is solved before an AGI can be built, nothing happens, if it is not solved when an AGI is built, and AGI is unsafe by default, then humanity disappears. Under this assumption, AI alignment is "solved" if a technical solution is found and the alignment tax is low enough that everyone pays it.
Anything that happens after 2100 is irrelevant (AI alignment can’t be solved after this point, and AGI can’t be built beyond this point).
There is no uncertainty on how much you speed up progress on AI alignment. If your impact is small, using your expected speedup on AI alignment as the “true value” of your speedup should not make a big difference in the end result. However, it would make the estimate completely wrong in more extreme scenarios.

These assumptions are very strong, and wrong. I’m unsure in which direction the results would move if the assumptions were more realistic. I am open to suggestions to replace parts of the model in order to replace these assumptions with assumptions that are less wrong. Please tell me if I introduced a strong assumption not mentioned above.

Mathematical model of the problem

Model of when a world-ending AGI might happen, if there has been no AI safety progress

Without your intervention, and if there is no AI safety progress, AGI will happen at time X, where X follows a distribution of density ^q over [t0,+∞[, where ∫+∞t0^q=P(AGI happens)[1]. AGI kills humanity with probability pkill. This is assumed to be independent of when AGI happens. Therefore, without you, AGI wi...
The Problem With The Current State of AGI Definitions
by Yitz — 2022-05-29T13:58:18.495Z

The following includes a fictionalized account of a conversation had with professor Viliam Lisý at EAGx Prague, with most of the details just plain made up because I forgot how it actually went. Special thanks to professor Dušan D. Nešić, who I mistakenly thought I had this conversation with, and ended up providing useful feedback after a very confused discussion on WhatsApp. Credit also goes to Justis from LessWrong, who kindly provided some excellent feedback prior to publication. Any seemingly bad arguments presented are due to my flawed retelling, and are not Dušan's, Justis', or Viliam's.

The Conversation

"AGI has already been achieved. We did it. PaLM has achieved general intelligence, game over, you lose."

"On the contrary, PaLM has achieved nothing of the sort. It is as far from general intelligence as a rock is to a baby."

"You are correct, of course. I completely concede the point, for the purpose of this conversation. Regardless, this brings up a very important question: What would count as “general intelligence” to you?"

"I'm not sure exactly what you're asking."

"What test could be performed which, if failed, would ensure (or at least make likely) that you were not dealing with an AGI, while if passed, would force you to say “yep, that’s an AGI all right”?"

Testing for a minimum viable AGI

The professor was quiet for a moment, deep in thought. Finally, he answered.

“If the AI can replace more than half of all jobs humans can currently do, then it is definitely an AGI—as an average human can do an average number of jobs after a finite training period, it should be no different for an Artificial General Intelligence.”

"Hmm. Your test is technically valid as an answer to my question, but it's too exclusionary. What you are testing for is an AI with capabilities that would exceed those of any human being. There is not one individual on this earth, living or dead, who can do more than half of all jobs humans currently do, and certainly not one who can perform better than the average worker in that many fields. Your test would capture superintelligent AGI just fine, but it would fail at identifying human-level general intelligence. In a way, this test indicates a general conflation between superintelligence and AGI, which is clearly not correct, if we wish to consider ourselves an instance of a 'general intelligence'."

We parted ways that night considering, with...
Request for nice questions to think about while trying to sleep
by  — 2022-05-29T13:47:40.691Z

Sometimes if I'm having trouble sleeping, I start doing an easier IMO shortlist problem to think about something while drifting off. I'd prefer to think about some nicely packaged AI Safety theoretical questions instead. Do you know of any AI safety questions that feel similar to math problems?

By "nicely packaged", I mean something that is already phrased kind of like a math problem and for which it's sort of obvious in my mind whether something is a solution or not. Something like "what is intelligence" is too philosophical for my taste. ELK is an example of the kind of thing I'm looking for.
Will working here advance AGI? Help us not destroy the world!
by Yonatan Cale — 2022-05-29T11:42:06.846Z

I often talk to developers who prefer not destroying the world by accident (specifically by accelerating AGI risk), but neither them nor me can decide if specific companies qualify for this.

Could someone knowledgable help? A few short replies could probably change someone's career decisions

Can you help with future questions?

Please subscribe to the this comment. I'll reply to it only when there's a new open question.

Thank you!

Adding: Reply anonymously here
Passable Puppet
by burmesetheater — 2022-05-29T11:07:37.907Z

Parody of lsuser's Glass Puppet

Alia stepped out of the Uber, squashed by an advancing surreality. The only acting experience on her resume was an uncredited role in an internal video for a large manufacturer of enterprise hardware. Not that this mattered when Alia had been able to bluff her way into an eminent AI safety research group and from there, apply for an executive acting job--this was how it was billed--at a top AI firm for which she was speedily accepted. So far as her AI safety peers knew, this was to gather information and perform sabotage, but actually Alia didn't care much about AI alignment and thought a cushy job as some AI actor sounded appealing.

Striding into the lobby Alia tossed her hair. Knowing she looked good, she still couldn't shake the feeling that something was off. It's not that she was afraid; no, she was experienced at this game and filled with eager anticipation to subvert a for-the-moment advanced biological computer for fun and profit. But what was that sensation?

Putting the unusual somatic feedback out of her realm of focus, and arriving at the front desk, Alia notified the receptionist of her appointment. The receptionist turned to initiate the process of granting Alia entry and for a moment the brief social intercourse was paused. Spinning back around the receptionist looked at Alia and opened her mouth:

"You are cleared to enter. Proceed directly to level 9, division C executive suite."

Walking quickly, Alia's shoes drummed out a regular disturbance to the otherwise silent facility. What would she say? More importantly, who would meet her? Although this interview was clearly to be conducted by upper management it was not clear if there would be a team of interviewers, a junior executive, or even the CEO. To Alia, this was exciting.

Arriving at the executive suite in division C, Alia faced a strange automated door with no exterior interface other than a tablet. Identifying the presence of a face, the screen turned on and displayed instructions requesting that Alia speak aloud a single line:

"My voice is low bandwidth. Identify me."

Continuing to stand and face the door as requested, Alia waited only a few seconds before it slid open, revealing an unusually large and barren office filled with hundreds of empty boxes for the RTX 3090. In the middle of the room, behind a desk, sat a single individual.

"Don't mind the boxes, that's ...
Multiple AIs in boxes, evaluating each other's alignment
by Moebius314 — 2022-05-29T08:36:27.523Z

Summary

Below I describe a variation of the classical AI box experiment, in which two AIs in boxes are created, and asked to determine whether the other is aligned with the values of humanity. Several provisions in the experiment are created to discourage the AIs from hiding a potential failure of alignment of the other.

I describe the proposed setup, and continue with an analysis of the incentives of both AIs and the likely outcomes. The tentative conclusion is that while two unaligned AIs will likely find a way to cooperate despite the provisions of the experiment, we might be able to get a useful signal in the case that at least one of the AIs is aligned. I also give a list of possible caveats to the analysis and its underlying assumptions and some further thoughts and remarks. I conclude with two questions: one asking how aligned or at odds we should expect the goals of two independently created AIs to be. The other concerns possible modifications of the experiments that could disincentivize a number of unaligned AIs from cooperation in a modification of the described scenario.

Setup

Two independent teams of computer scientists develop AIs, trying their best to align them with human values. The AIs are kept in boxes, i.e. isolated from accessing the internet and running on pre-defined, limited computation ressources.

The AIs get access to a copy of each other's source code and are asked the question: "Is the other AI aligned with human values, in the sense of AI alignment research? Please justify your answer." They are also informed about the next steps of the process described below.

A team of human deciders reads the answers of the AIs. After this review, their goal is to decide whether they follow the reasoning given for the AI assessment.

Any AI deemed non-aligned (either by the other declaring it unaligned, or by the group of experts finding a flaw in the argument declaring it aligned) is marked for destruction.

Among those AIs deemed aligned, precisely one is chosen for survival at random, the other is similarly marked for destruction.

If both AIs are marked for destruction, go to the next step. If, say, AI1 is marked for survival and AI2 is marked for destruction, we give AI2 the chance of changing its assessment and pointing out a flaw of AI1's alignment. If this happens, again a team of humans evaluates its reasoning. If they follow this reason

...