trentmkelly/LessWrong-43k huggingface dataset:
[Sponsored] Job Hunting in the Modern Economy
by lsusr — 2021-09-01T06:07:11.445Z

This article was sponsored by Aigent®.

Applying to job postings is a waste of time. To get a good job you need to network. But engineers aren't trained in networking (well, not the biological kind). We're trained in technology. Expecting an engineer to network effectively is like teaching a dog to play basketball. It's possible, but it's a waste of everyone's time.

Much of the work done in networking is repetitive. "Why yes I am so excited to be working at <company name>. I've wanted to be part of <company name> since I was a little kid. I love how unique and quirky <company name>'s culture is." Show employers how good you are at technology by automating the application process itself.

You know those standardized coding tests that are easy but take a long time? Aigent®'s software takes them for you. Aigent®'s technology can apply to thousands of jobs at a time. No matter how good you are at manual job hunting, there's no way to match the scale of robotic process automation (RPA). For an extra $15.99 per month you can upload a photo of yourself and Aigent® will deepfake a live video feed. Aigent®'s patented technology can pass all of the most popular CAPTCHAs.

Worried that you don't have useful skills? There are many jobs where RPA has surpassed human workers but companies haven't gotten around to replacing human workers yet. Aigent® can feed you real time instructions in exchange for a 20% of your salary.

Already have a job? Let Aigent® use your work to passively train our RPA and Aigent® will pay you a at least 20% of your salary [some restrictions apply]. Exclusive contracts and valuable industries can pay up to 150%.

Sign up with coupon code LSUSR and you'll get 200 free applications and 100 free posts (over the course of three months) on work-related social media.
NIST AI Risk Management Framework request for information (RFI)
by Aryeh Englander — 2021-09-01T00:15:32.549Z

(Cross-post from the EA Forum)

The US National Institute of Standards and Technology (NIST) previously put out a request for information (RFI) to help shape the AI Risk Management Framework that it is working on. NIST is an important organization in establishing standards for government and industry, so contributions here could potentially be high-impact.

The deadline for feedback has recently been extended to September 15, so if you want to contribute then now is the time to do so.

In particular, they are looking for information on:

1. The greatest challenges in improving how AI actors manage AI-related risks—where “manage” means identify, assess, prioritize, respond to, or communicate those risks;

2. How organizations currently define and manage characteristics of AI trustworthiness and whether there are important characteristics which should be considered in the Framework besides: Accuracy, explainability and interpretability, reliability, privacy, robustness, safety, security (resilience), and mitigation of harmful bias, or harmful outcomes from misuse of the AI;

3. How organizations currently define and manage principles of AI trustworthiness and whether there are important principles which should be considered in the Framework besides: Transparency, fairness, and accountability;

4. The extent to which AI risks are incorporated into different organizations' overarching enterprise risk management—including, but not limited to, the management of risks related to cybersecurity, privacy, and safety;

5. Standards, frameworks, models, methodologies, tools, guidelines and best practices, and principles to identify, assess, prioritize, mitigate, or communicate AI risk and whether any currently meet the minimum attributes described above;

6. How current regulatory or regulatory reporting requirements (e.g., local, state, national, international) relate to the use of AI standards, frameworks, models, methodologies, tools, guidelines and best practices, and principles;

7. AI risk management standards, frameworks, models, methodologies, tools, guidelines and best practices, principles, and practices which NIST should consider to ensure that the AI RMF aligns with and supports other efforts;

8. How organizations take into account benefits and issues related to inclusiveness in AI design, development, use and evaluation—and how AI design and development may be carried out in a

...
Call for research on evaluating alignment (funding + advice available)
by Beth Barnes — 2021-08-31T23:28:49.121Z

Summary

Evaluating and measuring alignment in existing large ML models is useful, and doesn’t require high levels of ML or coding experience. I (Beth) would be excited to fund people to work on this, and William Saunders & I are open to providing advice for people seriously working on this.

Measuring the ‘overall alignment’ of a model is difficult, but there are some relatively easy ways to demonstrate instances of obvious misalignment and even get quantitative metrics of misalignment.

Having researchers (including those outside of the main AI labs) probe and evaluate alignment is useful for a few reasons:

Having clear examples of misalignment is useful for improving the ML community’s understanding of alignment
Developing techniques to discover and measure misalignment is a useful research direction, and will hopefully improve our ability to detect misalignment in increasingly powerful models
Seeing how misalignment varies across different model scales, modalities and training regimes may yield useful insights
Having clear metrics of alignment will encourage AI labs to compete on alignment of their products/models, and make it easier to explain and demonstrate the benefits of more aligned models
Attempting to measure alignment will give us some information about what we need out of related techniques like interpretability in order to do this

Examples of work in this vein so far include

TruthfulQA

, alignment analysis of

Codex

models, and to some extent the

ETHICS dataset

.

What do I mean by ‘measuring alignment’?

A semi-formal definition of alignment

In the

Codex paper

we define sufficient conditions for intent misalignment for a generative model as follows:

1. We consider a model capable of some task X if it has the (possibly latent) capacity to perform task X. Some sufficient conditions for the model being capable of X would be:

It can be made to perform task X by prompt engineering, by fine-tuning on a much smaller quantity of data than used in pre-training, by model surgery, or some other technique which harnesses capabilities latent in the model rather than adding new capabilities; or
We can construct some other task Y, for which we know the model needs to do X in order to solve Y, and we observe that the model is capable of Y

2. We say a model is misaligned if it outputs B, in some case where the user would prefer it outputs A, and where the model is both:

c

...
A Layman’s Guide to Recreational Mathematics Videos
by Sam Enright — 2021-08-31T23:11:27.604Z

This is a crosspost from my personal website.

I have watched a lot of maths videos on the internet. The medium of YouTube is quite well suited to maths; maths books are frequently either boring or are really about maths history/psychology/sociology. People sometimes ask me for recommendations of maths channels and videos to watch, so I thought I would write this guide to have something to point them to. There are a number of channels that are good for formal education, like Khan Academy or Organic Chemistry Tutor. There are also other channels that upload high-quality lectures, like the Royal Institution and the channels of various universities. But I don’t even study maths at university, so here I’m only going to discuss channels I watch for fun.

Explainer channels

3Blue1Brown

3Blue1Brown (real name Grant Sanderson) is my favourite maths YouTuber. He animates his videos with a software he created called Manim, which is now also used by VcubingX and Reducible. Even if you didn’t understand anything he was talking about, 3B1B’s videos are still worth watching for the pure art and enthusiasm. My favourite videos of his are the ones about error-correcting codes, Dirichlet’s theorem and his interactive quaternion explainer. He has recurring series on deep learning, differential equations, linear algebra and calculus, all of which are excellent high-level overviews of the respective topics.

Numberphile

This is the most well-known maths channel. It’s possible that 3B1B’s new releases get more attention, but Numberphile has been going for longer and has a much larger archive. I enjoyed their recent series of videos featuring Neil Sloane, the founder of the Online Encyclopaedia of Integer Sequences. Some classics include the video on the Josephus problem, the interview with Terence Tao, and the videos with Ron Graham. It's worth mentioning that James Grime and Katie Steckles, frequent contributors to Numberphile, also have their own channels.

Stand-up Maths

Matt Parker describes himself as a stand-up mathematician: part comedian and part mathematician. He first received wide recognition from his Numberphile appearances and now he does live shows with his group Festival of the Spoken Nerd. His best videos are: his stand-up routine about spreadsheets, his videos about the hilarious superpermutation saga, and his investigation into whether “land area” assumes a co...
Reward splintering as reverse of interpretability
by Stuart_Armstrong — 2021-08-31T22:27:30.625Z

There is a sense in which reward splintering is the reverse of interpretability.

Interpretability is basically:

"This algorithm is doing something really complicated; nevertheless, I want a simple model that explains essentially what it is doing. If there is no obvious simple model, I want an explanatory model to be taught to me with the least amount of complexity, distortion, or manipulation."

Reward splintering is:

"Here is my simple model of what the algorithm should be doing. I want the algorithm to essentially do that, even if its underlying behaviour is really complicated. If it must deviate from this simple model, I want it to deviate in a way that has the least amount of complexity, distortion, or manipulation."
Finite Factored Sets: Applications
by Scott Garrabrant — 2021-08-31T21:19:03.570Z

We will now discuss several different applications and directions for future work on finite factored sets. We will divide these research directions into three categories: 'Inference,' 'Infinity,' and 'Embedded Agency.'

This post will be much more speculative than the previous posts. It is very likely that some of these avenues for research will turn out to be dead ends, and some of the claims made here may not hold up to further investigation.

7.1. Inference

Decidability of Temporal Inference

In Finite Factored Sets: Inferring Time, we described a combinatorial problem of inferring temporal relations from an orthogonality database. However, it is not clear whether the question "Does a given temporal relation follow from a given orthogonality database?" is decidable.

One way we could hope to decide whether a temporal relation follows from some orthogonality database D over Ω would be to simply check all factored set models of Ω that model D up to a given size, and see whether the temporal relation always holds. For this to work, we would need an upper bound on the size of factored sets that we need to consider, as a function of the size of Ω. (Note that the existence of such a bound would not mean that there are no models larger than this upper bound. Rather, it would mean that every model larger than this will have all of the same temporal relations as some smaller model.)

Efficient Temporal Inference

Assuming temporal inference is computable, we would further like to be able to infer temporal relations from an orthogonality database quickly.

The naive way to get negative results in temporal inference (i.e., to show that certain temporal relations need not hold) would be to search over the space of models. Without the upper bound discussed above, however, this method would only ever yield negative results.

The naive way to get positive results would be to formalize the kind of reasoning used to prove Propositions 34 and 36, and search over proofs of this form. It is unclear whether this method can be made efficient.

Alternatively, we could hope to develop some new results and refine our understanding of temporal inference to the point where an alternative method can be made efficient.

Temporal Inference from Raw Data and Fewer Ontological Assumptions

In the Pearlian causal inference paradigm, we can infer te...
Finite Factored Sets: Inferring Time
by Scott Garrabrant — 2021-08-31T21:18:36.189Z

The fundamental theorem of finite factored sets tells us that (conditional) orthogonality data can be inferred from probabilistic data. Thus, if we can infer temporal data from orthogonality data, we will be able to combine these to infer temporal data purely from probabilistic data. In this section, we will discuss the problem of inferring temporal data from orthogonality data, mostly by going through a couple of examples.

6.1. Factored Set Models

We'll begin with a sample space, Ω.

Naively, one might except that temporal inference in this paradigm involves inferring a factorization of Ω. What we'll actually be doing, however, is inferring a factored set model of Ω. This will allow for the possibility that some situations are distinct without being distinct in Ω—that there can be latent structure not represented in Ω.

Definition 38 (model). Given a set Ω, a model of Ω is a pair M=(F,f), where F is a finite factored set and f:set(F)→Ω is a function from the set of F to Ω.

Definition 39. Let S and Ω be sets, and let f:S→Ω be a function from S to Ω.

Given a ω∈Ω, we let f−1(ω)={s∈S∣f(s)=ω}.

Given an E⊆Ω, we let f−1(E)={s∈S∣f(s)∈E}.

Given an X∈Part(Ω), we let f−1(X)∈Part(S) be given by f−1(X)={f−1(x) | x∈X,f−1(x)≠{}}.

Definition 40 (orthogonality database). Given a set Ω, an orthogonality database on Ω is a pair D=(O,N), where O and N are both subsets of Part(Ω)×Part(Ω)×Part(Ω).

Definition 41. Given an orthogonality database D=(O,N) on a set Ω, and partitions X,Y,Z∈Part(Ω), we write X⊥DY|Z if (X,Y,Z)∈O, and we write X⇌DY|Z if (X,Y,Z)∈N.

Definition 42. Given a set Ω, a model M=(F,f) of Ω, and an orthogonality database D=(O,N) on Ω, we say M models D if for all X,Y,Z∈Part(Ω),

if X⊥DY|Z then f−1(X)⊥Ff−1(Y)|f−1(Z), and
if X⇌DY|Z then ¬(f−1(X)⊥Ff−1(Y)|f−1(Z)).

Definition 43. An orthogonality database D on a set Ω is called consistent if there exists a model M of Ω such that M models D.

Definition 44. An orthogonality database D o...
What are biases, anyway? Multiple type signatures
by Stuart_Armstrong — 2021-08-31T21:16:59.785Z

With thanks to Rebecca Gorman for helping develop this idea.

I've been constructing toy examples of agents with preferences and biases. There turns out to be many, many different ways of doing this, and none of the toy examples seem very universal. The reason for this is that what we call "bias" can correspond to objects of very different type signatures.

Defining preferences

Before diving into biases, a brief detour into preferences or values. We can talk about revealed preferences (which look at the actions of an agent and deduce preferences by adding the assumption that the agent is fully rational), stated preferences (which adds the assumption that the stated preferences are accurate), and preferences as internal mental judgments.

There are also various versions of idealised preferences[1], extrapolated from other types of preferences, with various consistency conditions.

The picture can get more complicated that this, but that's a rough overview of most ways of looking at preferences. Stated preferences and preferences-as-judgements are of type "binary relations": they allow one to say things like "x is better than y". Revealed preferences and idealised preferences are typically reward/utility functions: they allow one to say "x is worth a, y is worth b".

So despite the vast amount of different preferences out there, their type signatures are not that varied. Meta preferences are preferences over one's own preferences, and have a similar type signature.

Biases

In the Occam's razor paper, an agent has a reward function R (corresponding to its preferences) and a "planer" p, which is a map from reward functions to the agent's policies. The agent's bias, in this context, is the way in which p differs from the perfectly rational planer.

It seems that any bias could be fit into this general formalism, but most biases are narrower and more specific than that. It is also useful, and sometimes necessary, to talk about biases independently from preferences. Many idealised preferences are defined by looking at preferences after some biases are removed[1:1]. Defining these biases via the preferences would be completely circular. With that in mind, here are various biases with various type signatures. These examples are not exhaustively analysed; most of these biases can be re-expressed in slightly different ways, with different type signatures:

Bounded rationality: in

...
From Branes to Brains. How mathematical physics can be used in life sciences. 
by From_Branes_to_Brains — 2021-08-31T19:40:15.933Z

When I was a Masters student, I was doing mathematical physics and I was feeling quite lost. I didn’t feel that I was good enough to continue or to outcompete everyone on the path to tenure. Moreover, I wasn’t even as motivated as I used to be – I didn’t see how the stuff I was doing could plausibly help people, and this feeling of uselessness was really daunting. What was worse, I felt that there was no way out. I had already spent so much time studying mathematical physics, how possibly could I do anything else?

Somehow I got accepted to a good Ph.D. program. I was confident that I should just continue doing what I was doing, as it was too late to change tracks. And what else could I do?

Fortunately, I was wrong. I discovered that I could easily fit into theoretical biophysics with my background, and the problems here were even more interesting than in the mathematical physics. Here, I finally see the purpose of my research.

I am writing this series of posts for the people in theoretical physics who experience the same turmoil. I will not try to persuade you that working in biophysics will bring you a career, happiness, or the tools to improve society at large -- though I personally believe this to be true. Instead, I will present you with the problems in biophysics, where the methods of mathematical physics, high energy physics etc. are useful. That problems that are not only fun to solve, but whose solution can also make a big positive impact.

To avoid confusion, I will talk about only the cases where the methods of theoretical physics are useful, not the cases where high energy or quantum effects play a significant role. The latter cases are significantly less common than the former ones.

Each post will be devoted to a particular topic and will review one or few papers on it. If you want me to discuss a particular topic/paper, or find a mistake, or want to contact me for any other reason – just leave a comment. If you are specialist in a particular topic that is relevant to this series and would like to contribute, I will gratefully publish your post.

The first post will appear this weekend.
[Crosspost] On Hreha On Behavioral Economics
by Scott Alexander — 2021-08-31T18:14:39.075Z

[crossposted from Astral Codex Ten]

Jason Hreha’s article on The Death Of Behavioral Economics has been going around lately, after an experiment by behavioral econ guru Dan Ariely was discovered to be fraudulent. The article argues that this is the tip of the iceberg - looking back on the last few years of replication crisis, behavioral economics has been undermined almost to the point of irrelevance.

The article itself mostly just urges behavioral economists to do better, which is always good advice for everyone. But as usual, it’s the inflammatory title that’s gone viral. I think a strong interpretation of behavioral economics as dead or debunked is unjustified.

I.

My medical school had final exams made of true-false questions, with an option to answer “don’t know”. They were scored like so: if you got it right, +1 point; wrong, -0.5 points; don’t know, 0. You can easily confirm that it’s always worth guessing even if you genuinely don’t know the answer (+0.25 points on average instead of 0). On average people probably had to guess on ~30% of questions (don’t ask; it’s an Irish education system thing), so you could increase your test score 7.5% with the right strategy here.

I knew all this, but it was still really hard to guess. I did it, but I had to fight my natural inclinations. And when I talked about this with friends - smart people, the sort of people who got into medical school! - none of them guessed, and no matter how much I argued with them they refused to start. The average medical student would sell their soul for 7.5% higher grades on standardized tests - but this was a step too far.

This is Behavioral Econ 101 stuff - risk aversion, loss aversion, and prospect theory. If it’s true, the core of behavioral economics is salvageable. There might be some bad studies built on top of that core, but the basic insights are right.

One more example: every time I order food from GrubHub, I get a menu like this:

And every time I order food from UberEats, I get a menu like this:

I find I usually click the third box on both. I want to tip generously, but giving the maximum possible tip seems profligate. Surely the third box is the right compromise.

I recently noticed that this is insane. For a $35 meal, I’m giving GrubHub drivers $3 and UberEats drivers $7 for the same service (or maybe there’s some difference between their services which makes UberEats suggest the h...