Tuesday, December 27, 2011

How not to publish law school employment statistics

Here's what a prospective student who is considering the University of Washington will find on the school's web site when trying to figure out whether the school's current $26K resident and $40K non-resident (unlike a lot of state schools, non-residents can't get residency UW after a year) price tag is worth it:

"96% of the Class of 2010 graduates (169 out of 176) were employed as of February 2010 [this is surely a typo, which is supposed to read February 2011]. Employed does not include two (2) graduates enrolled in graduate programs. Three (3) students remained seeking employment and two (2) students were not seeking employment."

Note this page was last updated two weeks ago, so this is the school's considered response to the criticisms from the law school transparency movement that law school placement and salary data is misleading.

UW's response, in short, is to continue to trumpet a completely phony overall "employment" rate, while providing as close to no information as possible about everything else. Pretty much the only meaningful piece of information on this page, from a practical perspective, is that only 43% of "employed" 2010 UW grads had jobs of some sort in private law practice nine months after graduation.

How many of the other 57% of "employed" grads had jobs that required a law degree? How many of those jobs were temporary, or part-time, or both?  For that matter, how many of the law firm "jobs" were in one or both of those categories?  What did the 15% of jobs described as "clerkships" involve?  (Remarkably, USNWR reports that 18.0% of the UW Class of 2009 had federal Article III judicial clerkships.  This number attributes to the school the fourth-highest percentage of federal judicial clerkships for any law school, behind Yale, Stanford, and Harvard -- Harvard's listed percentage is 18.1%. Numerous attempts by the Law School Transparency project to get the school's career services office to confirm or correct that number have failed, and it's still featured in the latest available USNWR data.).  How many UW grads were "employed" by the school itself, or in school-funded positions?  How many of the 43% of its "employed" graduates who had positions with law firms were in solo practices, or in firms with two to ten employees -- a category which includes things such as doing temporary contract work for solos, two graduates forming their own firm etc?

Not least of all, what percentage of graduates had known salaries, and what were they?  Perhaps I'm missing something on the UW site, but I can't find any of the above information on it.  All it tells me is that 96% of UW's graduates are "employed" nine months after graduation.

If I look elsewhere, I can find this.  (Given that this "informational" page includes the stats for UW's Class of 2014 it can't be more than a few months old).

In recent years, Washington has continued to demonstrate tremendous ability to place its students into some of the top law firms and judicial clerkship positions of Washington state and the Northwest.  Of the class of 2010, 96% of graduates were employed within 9 months of graduation, roughly 69% of whom secured employment within the state of Washington.  43% of graduates accepted jobs with private law firms, with a median starting salary of $95,000, while 15% took on judicial clerkships.  Outside of the Northwest, job prospects are predictably less stellar for Washington students, although the school’s alumni network allows students the opportunity to pursue employment in all corners of the nation. 
Are we to assume from this that UW had salary information for 100% of the 43% of its employed graduates who were working for law firms? That seems highly improbable, given that, according to what it reported to USNWR, for its 2009 class UW did not have any salary information for nearly a quarter of its graduates employed by law firms, and on a national level the percentage of graduates not reporting salaries rose in the class of 2010.

It's instructive to compare all this to the new employment and salary data which was just made public by Ohio State.  While not perfect, the OSU data allows a prospective student to answer almost all the questions listed above within a tolerable degree of accuracy.  This is a particularly salient comparison, given that UW and OSU have very similar USNWR rankings (30th and 35th) and tuition rates, and that they are both located in major metropolitan areas of states with very similar overall unemployment figures (8.7% and 8.5% in November, respectively).

There's a special irony in UW's numbers, which is that the UW law library has a deserved reputation as a leader in compiling information about legal academic publishing.  If you want to find out what law professors published in law reviews last week, UW is a great place to go for detailed information. On the other hand, if you want to find out what percentage of the school's graduates are actually employed as lawyers of some sort nine months after graduation . . .


  1. I get your overall point, but why are you writing as if being a solo practioner or working for a firm of ten or less is not employment?

  2. LawProf is a finalist at ATL's LotY contest. Let's vote him up.


  3. It's extremely difficult for a new graduate to make a living as a solo. It's a saturated market, and new grads are at a big disadvantage because of the whole "law school doesn't actually teach you how to be a lawyer" thing.

    As for very small firms, a lot of the new grads "employed" by such firms don't have real positions -- they're doing contract work or clerking. Check out the percentages of grads at such firms that report their salaries. It's minuscule.

  4. Both Ohio State and U. Washington should be shut down. Why aren't lawyers seizing control of the ABA accreditation process? Leaving it to the law school cronies is irresponsible.

  5. That may be true, but I have friends who have been (are) in small firms who might wonder at being described as "employed".

  6. Yes, there are also lots of people "employed" by 501+ firms (via temp agencies) doing doc review. They don't have real lawyer jobs either. Once you exclude everybody in such positions what percentage of the 51% of 2010 law school grads who were listed as working for law firms in February 2011 actually had real jobs?

  7. That you could probably find out-- the part about how many lawyers come to firms through temp agencies. It would take some actual work, though.

  8. The stats and the way they have been broken down by many law schools are way off. My school's info is as bad as Washington's listed above.

    These schools will tell you with a straight face that they have gone above and beyond what is required of them by the NALP and/or the ABA when reporting their numbers but as noted, the numbers do not really tell you a thing but rather hide the ball.

    It seems that only the best schools come clean with their numbers (Yale, Chicago). While improvement is still needed in those numbers, they are so much better than the numbers reported by average schools like Washington.

  9. Also, what does U. Washington mean by "business"? In-house at Starbucks, or serving coffee at Starbucks?

  10. "43% of graduates accepted jobs with private law firms, with a median starting salary of $95,000"

    This technique of using biased samples to calculate descriptive statistics for a population is really outrageous fraud by law schools.

    It is the equivalent of me taking the following 25 law school grades:
    5 As
    5 A-s
    5 B+
    5 B
    B B-

    And reporting it on my resume as a GPA of 3.84 by averaging only the As and A-s. Even the schools that tell you how many students were included in their average salary numbers (like Ohio State) are committing fraud, because that would be like me writing "GPA of 3.84 based on 10 reported courses."

    They know the subset of students that they use to calculate the average salary does not represent the population, just like I would know that the 3.84 earned in those 10 classes does not represent my overall GPA. Despite this knowledge, they communicate the false statement hoping to mislead people.

    If I pulled that stunt with my GPA I could very well be reported to the bar. So why are law school allowed to get away with it?

  11. I think that much of the discussion here has a tendency to interpret everything using the least favorable perspective; perhaps that is needed to counteract other biased coverage, but this time I felt a bit sensitive because you are talking about my school!

    UW does have pretty good clerkship placement -- partly because we send a fair number of people up to Alaska, to Montana, Oregon, etc. There are only two other Washington law schools, and UW also has a relatively small class size so there's less competition for clerkship-type positions. I think my class has about 175 students. In fact, a handful of my classmates accepted federal appellate clerkships for next year; not all of them were on law review, but there are also lots of unique and intelligent people who don't do that route and focus their efforts in different ways. (Yeah, yeah precious snowflake language, I know.) I think it is still really challenging to find paying public service jobs, but I do keep hearing people talking about jobs at various govt agencies and the military.

    I think the employment picture is more complex than "96% employment at 9 months" or "90% of graduates must be living in a ditch somewhere and UW should be shut down." For instance, I wasn't on any journals or have top 10% grades, but I have one of those 160k job offers. Not that that should be defined as success, but it's not bad, either, and it's a job I wouldn't have had access to if I hadn't gone to UW. Same deal with a bunch of my fellow non-law review friends. And I have friends that are working part-time at small firms with fewer than 10 attys, who will continue after graduation. Some of those are bad jobs, and some of them are good jobs that are not temporary.

    I'm sure that a fair number of my classmates are regretting their decision to go to law school, but at least I'm sure that no one is graduating with $200k in debt.

  12. 1:18, Asking for statistical analysis of the entire population - as opposed to a tiny subset that is fraudulently used to represent that population - is not interpreting things in the least favorable manner. That is interpreting things in the most neutral and complete manner.

    What you just did, with that nonsense about how UW places many in federal clerkships and 160k jobs, that's interpreting things in an extreme manner. We don't need intentionally vague and misleading terms like "many." We need objective and accurate numbers.

  13. Alright, sorry 1:18 I'll stay away from your blog.

  14. 1:18: Thanks for stopping by. Care to comment on UW continuing to report that 18% of the 2009 class had Article III clerkships?

  15. @1:18. I emerged from undergrad debt free; after three years at UW Law, I'm $200+ in debt and without gainful employment. Excellent references, amazing resume, stunning good looks, etc.

  16. That's awful 3:37. How did you break 200k?

  17. Oh never mind it must be interest accruing during unemployment.

  18. @ LawProf -- I am a student, so I don't have all the data on student outcomes, and I don't know anyone from the class of 2009. But aside from the things I mentioned earlier about geography, maybe judges out here are just less fixated with Ivy League grads? I suppose some of them graduated from UW. There are also a couple of faculty members who coach interested students one-on-one on how to navigate the clerkship application process. We got an email in September mentioning that five members of the class of 2012 had accepted federal appellate clerkships, with no comment on trial level federal clerkships. I do know several people who have accepted various types of state clerkships (trial, appellate, and supreme court); I assume that those categories aren't included if they're reporting a group as Art III clerkships. I guess these all seem like pretty discrete categories that are not amenable to exaggeration, so I'm inclined to believe the info being reported about clerkships. That's in contrast to salary/private employment stats, which you can misrepresent by not mentioning % reporting, details about the position, and other things discussed in detail on this blog. I am one of those people who went to law school as a bit of a last-minute decision without knowing any lawyers or much about the field beforehand, and I was shocked to realize sometime during 1L year (in 2009) that you couldn't trust much of the employment statistics that all law schools publish.

    I don't mean to speak as any kind of official representative. I'm just a middle-of-the pack, grades-wise, student, and from my perspective I see a lot of similar classmates who have good jobs lined up and some who are still looking. I guess I just didn't want everyone reading this to think that most people have had a miserable outcome, although I'm sure there are a fair number of people who are really unhappy with their prospects. Sorry, 3:37 -- that sounds really rough.

    On a related note, all of the associates I've met (since I started meeting lawyers after beginning law school in 2009) who have since left their firms have gone in-house or to other firms of similar size. I realize this is a really small sample size, and maybe I haven't met anyone who's been laid off because it's not 2008 anymore. But one of the reasons I read this blog is because it's such a different perspective. On Above the Law, you'd come away thinking every lawyer practices in NYC and flips out if she gets a bonus less than $30k. On this blog, every lawyer is miserable and/or unemployed. In real life, things just seem a lot more mellow and ok -- like how my friends who are working at small firms are happy about it and like their coworkers a lot, and other friends are excited about their state court clerkships, although the way those positions are sometimes described here, it almost seems as if they should be disappointed in themselves. It's hard to reconcile everything. Sorry about the huge post.

  19. Will they be forced to disclose more detailed information next year when the new rules take effect, or did that not happen?

  20. The bottom line is that it is tough to find a good legal job out there right now, and frankly, there are just too many lawyers.

  21. I've been catching up on some reading after a hiatus over the holidays. I was surpirsed to see the content of this post, since it's about my class!
    I think there are a few factors that need to be considered when looking at UWs numbers that may be overlooked here. First, UW is the main law school for the Seattle market which has a fairly tight-knit legal community, many (most?) of whom are UW grads.
    Second, UW undeniably benefits from geography. UW is usually regarded as the best school in the northwest, so UW grads get jobs throughout much of the region. I personally know of about a dozen of my classmates who went to Alaska, Idaho, and Oregon to practice and are happily employed (for a class of 175, that's a fairly large percentage). This also helps in the clerkship department, as UW grads are often applying for clerkships in both state and federal courts in multiple states that frequently have judges or other clerks with UW connections.
    Third, I am not aware of any program whereby the school hires graduates in temporary jobs to inflate the numbers. This doesn't mean it doesn't exist--just that I've never heard of it nor known anyone who took part in it if it does exist.
    All of this is not to say that the employment figures are perfect, and from the anecdotal evidence I've heard, it appears that the salary figures only reflect a subset of the class. Still, my class is doing pretty well, all things considered, and it is my personal opinion that these numbers are not grossly distorted. They could undoubtedly be improved, but they don't seem fraudulent to me.
    From what I've heard, the same may not be true about UW's class of 2011, but overall, the class of 2012 seems to be doing alright.

  22. *class of 2010 seems to be doing alright

  23. Nice weblog here! Additionally your website rather a lot up very fast!

    What host are you the usage of? Can I get your affiliate link on your host?
    I want my site loaded up as quickly as yours lol
    Feel free to visit my webpage : I loved this


Note: Only a member of this blog may post a comment.