Partnership or death?

An invariable truth about legal academic life is what is known as the Wanker Principle: Any law school rating system designed by a law professor exists primarily for the purpose of trying to make that professor's current law school look good.

This truth is vindicated in a particularly amusing way by a new article just published by Loyola Los Angeles law professor Ted Seto, Where Do Partners Come From?  The article is a fairly brazen exercise in intellectual bad faith, starting with its claim that, "the purpose of the present study is not to supplant or critique existing rankings."  In fact the purpose of the study is to advertise in its first and most prominent table that Loyola Los Angeles is a top 25 law school in regard to producing big firm partners, and of those 25 schools it's the most "under-rated" relative to its U.S. News ranking, although Seto possesses just enough tact to avoid pointing this out directly:

Readers may find some of these numbers surprising . . .St. John’s, a school only slightly larger than the U.S. average, outperforms its U.S. News ranking by an astonishing 53 places, Miami by 51 places, Villanova by 49, DePaul by 47, Catholic by 43, Loyola Chicago by 42.

Seto's study's definition of "performance" is, shall we say, not methodologically nuanced: he totals up how many big firm partners graduated from individual law schools, and then creates a chart of "Top 50 Feeder Schools" in regard to producing BigLaw partners.  This of course makes much of his "Top 50" list simply a proxy for a law school's size, since he doesn't control for this overwhelmingly important confounding variable.

Here's Seto's explanation for this methodological choice:

Many rankings—U.S. News, among others—compare schools predominantly on a “per capita” or “per student” basis. The premise is that schools whose average students (or professors) are better should be ranked higher. This may make sense if one’s goal is to establish a Platonic hierarchy. Theoretical rankings, however, are often of only indirect relevance to real-world decisions. Economies of scale exist in law firm hiring, as elsewhere. If employers cared solely about per capita outcomes, they would all interview at Yale (ranked No. 1 by U.S. News). They don’t. For employers attempting to allocate scarce recruiting resources, aggregate numbers matter.

This seems like a long-winded way of saying that all other things being equal an employer would rather interview at a big law school than a small one. But all other things aren't equal, and indeed in regard to the specific employment outcomes Seto's article is discussing, Yale's small size has exactly zero relevance to the analysis, since every single firm that's part of that analysis interviews at Yale.

Things then get even more confusing:

Whether and when per capita data should be relevant to applicants is a more complex question. The single most important determinant of how schools perform on most outcome measures (bar passage, hiring, big-firm partnership, etc.) is the quality of the students they attract. In significant part, therefore, per capita outcome measures are merely proxies for student quality. Unfortunately, applicants commonly misread such measures as reflecting the value added by attending one school rather than another. (“I am more likely to pass the bar if I go here rather than there, because the bar passage rate here is higher.”) Unless a measure controls for student quality, however, it says nothing about the value likely to be added to a particular student by a particular school. The fact that students at highly ranked schools almost always pass the bar is largely a function of the native ability of the students themselves. It does not necessarily mean that such schools do anything to prepare students for the bar—indeed, the fact that students at more selective schools are likely to pass the bar in any event may even reduce pressure on such schools to pay attention to bar preparation.

Analysis of the value added by particular schools with respect to particular output measures is a project beyond the scope of this article. I have not attempted any such analysis here. What I do offer are the raw numbers—which I believe are less likely to mislead.

I confess to being unable to make heads or tails of what Seto is arguing here.  What relevance does the (plausible) claim that graduate employment outcomes may be largely if not wholly a function of the initial quality of matriculants, rather than of any value added by law schools themselves, have to a claim that it's better to offer raw numbers in regard to employment outcomes rather than adjusting for law school size? As far as I can tell the two issues have nothing to do with each other. (Indeed if you don't adjust for law school size, the Thomas M. Cooley School of Law ranks ahead of Yale and Stanford in regard to how many of its graduates get jobs as lawyers. Etc.).

Anyway, in a footnote Seto points out that those so inclined can convert his raw numbers to per capita outcomes.  Doing so reveals that Seto's analysis is misleading to prospective law students -- who, let's face it, are the real target audience of this exercise -- in a way that goes far beyond the relatively trivial fact that he's managing to make his law school look comparatively good.

Seto assigned ten law students the task of looking up the graduating law schools of every partner at an NLJ 250 firm who graduated between 1986 and 2010.  He then limited his analysis of this data to the NLJ 100 (if you want to extrapolate the percentages below to the NLJ 250 you can increase them by roughly one-third).  Now obviously anyone who graduated after 2001 or so had not yet had a chance to make partner at these firms, so in what follows I'm going to exclude the classes of 2002 and following from the analysis.

Seto's researchers determined that 16,799 ABA law school graduates who graduated in 1986 or later were partners at NLJ 100 firms as of 2010.  In my view, an even more misleading aspect of Seto's analysis than his refusal to adjust for law school size is his failure to provide any baseline for readers in regard to what this number represents when compared to the overall cohort of law school graduates.  Fortunately, this omission is easy to correct:

Total JDs awarded by ABA-accredited law schools 1986-2001:  610,160

Total partners in NLJ firms in 2010 who graduated in 1986 or later:  16,799

Percentage of ABA-accredited law school graduates who graduated between 1986 and 2001 who were partners with NLJ 100 firms in 2010:  2.75%.

Note that Seto's article is framed as an exploration of an important factor prospective law students should keep in mind when comparing law schools (He begins with an anecdote about a potential Loyola law student whose career ambition was to become a big firm law partner in Los Angeles, who foolishly went to Vanderbilt instead).  But Seto goes out of his way to avoid discussing or even acknowledging that the odds of any prospective law student becoming a big firm partner are terrible, and indeed remain quite long even if the 0L should find him or herself ascending into the ranks of the Blessed 14.  

How terrible? Seto's analysis features a 25-year window.  If we assume the average age of law school graduates is 28, the average law school graduate has about a 5.6% chance of dying before his or her 25th-year law school class reunion.  In other words, the average law graduate's chances of failing to show up at that reunion on account of being dead are more than double than the odds the graduate will have the opportunity to attend while holding a partnership with a big firm.

At how many law schools does the average graduate have a better chance of being a partner at a large law firm 25 years after graduation than of being dead?  The answer is, the T-14 plus four.

 

Harvard:         10.1%

Yale:                7.7%

Stanford:          8.2%

Columbia:        7.3%

Chicago:         13.4%

NYU:              7.1%

Michigan:         7.4%

Penn:               7.5%

UVA:               8.8%

Duke:               6.5%

Berkeley:          5.9%

Northwestern:   8.0%

Cornell:             6.5%

Georgetown:     7.1%

Texas:               6.2%

Vanderbilt:        5.8%

Boston U:         7.2%

Illinois:              5.9%

As for the two schools that, per Seto's analysis, supposedly outperform their U.S. News ranking by more than 50 slots, they actually have worse numbers than would be predicted by their U.S. News rank alone.

St. John's:       2.7%

Miami:            2.4%

As does, to choose another school at random, Loyola LA, which is currently ranked just below the top quarter by U.S. News, but which it turns out is a slightly worse than average law school in regard to producing big law firm partners (2.7%).

I have a few more things to say about Seto's article but I'll save them for another post.