This study is a good example of how hard it is to analyze a complex problem. In an effort to establish controls, the study is based on a specific methodology. The authors mailed applications to top law firms and measured the response rate. The resumes were for fictional students from second-tier law schools, identical except for gender and class markers.

> All applicants were in the top 1% of their class and were on law review, but came from second-tier law schools. This was important because graduates from the most elite law schools (e.g., Harvard and Yale) are typically recruited on-campus.

The study identifies quite solid evidence of discrimination against those candidates. However, it then draws a pretty broad conclusion:

> Our findings confirm that, despite our national myth that anyone can make it if they work hard enough, the social class people grow up in greatly shapes the types of jobs (and salaries) they can attain, regardless of the achievements listed on their resumes.

Although the conclusion is probably true, the study doesn't really adequately support it. The authors studied responses to mail-in applications to "top law firms." But top law firms hire almost all of their candidates from on-campus interviews at top law schools. At every law school I am aware of, law firms select students to interview through an semi-anonymous process were names and gender are hidden. (Obviously, the law firms find out the name and gender during the interview itself.) Indeed, at top law schools, firms don't even get to select candidates in the first instance. Interviews are instead allocated through a bidding/lottery system.

Additionally, the applications were identical with respect to grades and law school rank. But those are the main criteria law firms use to select entry-level hires. Keeping them constant isolates the effect of other resume factors, but also strips out information about the relative impact of class signals versus grades and law school rank. Therefore, it cannot distinguish between a system that is mostly meritocratic except when it comes to tie-breakers, and one where class can play a large role even overcoming differences in grades and law school rank.

(I will also add that some of the markers on the resume are not necessarily markers of class, but other things. Law is an overwhelmingly liberal profession, for example, while listing that you enjoy "country music" on your resume could be seen as signaling conservative political attitudes.)

So the study draws conclusions about how law firms as a whole work, but based on studying a hiring pipeline that doesn't reflect how the vast majority of candidates are hired--one that reflects the exception not the rule. For example, one might conclude from the article that entry level attorneys at top law firms are mostly men, but in fact the pipeline is essentially 50/50 until you get to the senior associate level.

That is not to cast doubt on the conclusions in the study. Certainly, the feedback about stereotypes about women ring true as a matter of my anecdotal experience. (My wife was told, when she was going through interview season, that she didn't need to worry about it as much because I had already been offered a position at a good firm.) But it just shows how hard this stuff is to study. The study generalizes beyond what is supportable by its methodology. And it bypasses a mechanism--blind/lottery on-campus interviews--that the profession deliberately implemented to reduce bias in hiring.