Saturday, December 17, 2011

Social change and the politics of law school

Philip Hart, (Michigan Law School, Class of 1937)  a great man who was known as "the conscience of the Senate" -- his colleagues voted unanimously to name the Senate office building after him -- once said that when he came to Washington his goal was to save the world.  After a time he decided it would make more sense to limit his ambitions to trying to save his country.  Eventually, he said, he was just trying to save the dunes.

Politics, Max Weber remarked, is always "the slow boring of hard boards."  Serious social reform is always a difficult, slow, and frustrating process, which rewards its proponents with many more defeats than victories.  That reform happens, roughly speaking, in two ways: on the structural, institutional level, and on the personal, individual level.  On this blog I've been emphasizing the former, which isn't to say that the latter isn't important as well (indeed the former would never happen without the latter).


DJM asks:

What would happen if every law professor (or even, say, 65%?) stepped up to the plate and devoted him or herself seriously to redesigning their courses in a way that would produce excellent professionals? What if those professors were willing to read the literature on professional development and to talk to other experts in the field? What if they were willing to treat professional education as a subject itself worthy of serious study and discussion?

And what if those professors were also willing to talk to one another about how their courses might fit into an integrated professional education? And if they also met with practitioners to discuss ways in which legal education could bridge into training during the early years of law practice? And thought of creative ways to integrate willing practitioners into legal education?
 These are good questions. The answers, I think, are that legal education would be improved significantly if something like this were to happen, both because the substantive quality of what goes on in law school classrooms would improve as a result, and because even asking these sorts of questions in a serious way would lead to further questions about how legal education could be restructured to be much more cost-effective and life-enhancing than it currently is.

The difficulty is that reform at the individual level, while always very important as a spur to social change, can only go so far.  Consider an analogy from the world of politics.  If one believes that the vast and rapidly growing gap between the rich and the poor in America is a moral outrage, one could exhort rich people to behave better at the individual level, by for example paying their workers higher wages, and contributing more to charity. And it's important to do this -- although not, primarily, because some people will listen to such exhortations and act on them.  A few will, but the bigger value of such messages is that they can spur change at a structural level, by putting more and more pressure on the government to do something about the extent to which American society has slid all the way back to Gilded Age levels of economic inequality.

I believe the same is true within the world of law school reform.  It's important to exhort legal academics to behave better: to, for example, get us as individuals to work harder to make sure our classes are valuable both intellectually and vocationally to our students; to convince us to fight within our institutions to try to lower or at least freeze tuition; and to encourage us to demand that our employers publish genuine employment and salary data, and to not take no for an answer when they refuse.

All this is important, and indeed imperative. Still in the long run trying to bring about reform at the level of individual behavior is a means to a greater end, which is to spur institutional and structural reform. Individual change can only go so far, because, for individuals as individuals, the institutional and structural incentives are, at present, all wrong. Again, to analogize to the larger political world, if families who have incomes of $40,000,000 per year pay a lower percentage of their total income in taxes than families with incomes of $40,000 (as is actually the case in the USA at present) then all the moral exhortation in the world isn't going to do much, in a direct way, to lessen radical economic inequality.  But what that exhortation can do is help move the political process towards the kind of structural reform that will alter basic economic relations toward a more just state of affairs.

One of the ways in which it can do that is by inspiring people within the group that currently benefits disproportionately from the status quo to work to bring about change.  If such change depended solely on the ability of those people to convince the group to change the status quo to the group's (at least temporary) disadvantage, then the change would almost certainly not happen.  But luckily, change within a privileged group comes about from a combination of internal and external pressure.

What is needed are leaders within the group who will increase the pressure for change both within it and within society in general.  If you are a legal academic, you have the choice to be one of those leaders: to fight for institutional reform, fiscal responsibility, and justice for our students.  If you do so you will, I suspect, find allies in unexpected places.  Perhaps there will be very few of you at your particular institution, at least at first, It's even possible that you'll have to begin more or less alone.  But a leader is by definition and by nature the one who goes first.

82 comments:

  1. the new green background makes it harder to read, just sayin'

    ReplyDelete
  2. p.s. good luck finding leaders amongst the crop of losers who couldn't get jobs . . . and among law professors whose salary comes by continuing the scam.

    ReplyDelete
  3. To be sure, the law school system needs some reforms. But pronouncing the entire system a "scam" goes well beyond mere exhortation. I've jumped through a lot of hoops to get where I am, and as noted earlier, this is not a zero sum game.

    ReplyDelete
  4. No one said the entire system was a scam. Society obviously needs lawyers, but not as many as produced by the low tier schools that people like nyc teach at.

    ReplyDelete
  5. As noted earlier, for current law students it's negative sum collectively speaking. It takes a rather remarkable level of self-absorption to imagine that the number of hoops you've jumped through to get to where you are has any relevance to that fact.

    ReplyDelete
  6. Any way we could go back to the old format? It was much easier to read.

    ReplyDelete
  7. I agree with all LawProf says, and most definitely reform of legal education is needed. However, I refocus attention on a more dramatic reform that is urgently needed and long overdue. From a cost benefit analysis, for efficient use of society's resources, and for justice for students in the sense that students who spend three years studying more vocationally useful courses should have a reasonable expectation of getting a job to use at least some of the information learned and actually make a living in the long term using at least some of the information learned, there is one urgent, compelling reform needed. That urgent and compelling reform, more critical than modifying academic content and means of instruction, is simply radically reducing class sizes, by at the very minimum half. Any reform short of stopping the flood of far too many entrants, a large percentage of whom have crippling student debts, into an already overly saturated field, does little to address the basic problem. I emphasize the data from the Bureau of Labor Statistics demonstrating about 769K lawyers working as lawyers, the data demonstrating about 575K holders of JDs not working as lawyers (likely a sizable number of whom would like to find a decent lawyer job), the data demonstrating that less than half of recent years graduates (2010 in particular) obtained a permanent first job that requires a JD, and the federal government projections demonstrating that law schools are currently graduating about two new graduates annually for every new lawyer job that will likely be created annually for the next decade. I also reiterate anecdotal evidence demonstrating hundreds of thousands of underemployed and unemployed lawyers who have previously worked. These data clearly demonstrate that one dramatic reform is urgently needed, and that reform is long overdue...drastically reduce the number of JD graduates.

    ReplyDelete
  8. As much as I'm cynical and as much as I complain about the self-interest of the professors, I do believe that every professor is trying to create good students who will improve the world. The problem is that there's a great amount of disagreement about just what change is needed and how it should be accomplished. So without resorting to a dictatorship where at least one person gets his way, we're kind of stuck wringing our hands and asking these questions.

    ReplyDelete
  9. 3:53 what proposed changes do you disagree with?

    ReplyDelete
  10. Out of curiosity, is there such thing as a school that teaches leadership skills, or is that something that either innately exists or does not?

    ReplyDelete
  11. "No one said the entire system was a scam."

    The title of this blog is a bit of a hint. Besides, there are plenty of successful graduates from lower ranked schools. One of the wealthiest attorneys I know went to Brooklyn Law School.

    I went to a much higher ranked school than him and I don't make anywhere near the money he makes. The only people I know who do better are partners at major firms. What do they all have in common? They all had to jump through a lot of hoops to get where they are.

    ReplyDelete
  12. nyc,

    Can you do me a favor - can you rigorously break down your arguments step by step?

    For example, why would it be false to call something a scam if only X% (X =/= 100) of its actors are scamming?

    Or, how is the fact that one graduate of Brooklyn Law School made a lot of money proof that Brooklyn Law School is not a scam?

    Or, how is the fact that by "jumping through hoops" (whatever that means), some people became successful proof that anyone who wants to be successful can achieve it by jumping through hoops?

    ReplyDelete
  13. Very true and eloquent, LawProf--hear, hear! I hope lots of potential leaders, both inside and outside the academy, are out there reading. For those who are: This is a crisis with room for many responses. There are immediate, concrete steps like transparency; that change could open the door to many other reforms. There are bigger changes to be made within existing schools and practice organizations, as well as systemic, out-of-the-box reforms to envision. Changes can come from academia, practice organizations, bar associations, and individuals in any of those places. Fixing law practice is a problem with many handles to grab.

    But it's a good time to acknowledge that this blog and LST have already had significant impact. There has been impact system-wide, by increasing public awareness and securing the ABA reforms on employment reporting. On an individual level, they've kept me (and I hope others) pushing at our institutions. OSU's new employment data page went up yesterday. It's not perfect, but it shows percentages employed in PT or temporary jobs, percentages employed by the law school, and other important data (you have to click "show table" for some of the info. For next year, we're aiming for more salary data and more granular information about types of jobs (i.e., what are people doing in these jobs, even if FT with JD required?). But the new page gives more info to prospective students and--equally important--should provoke more faculty discussion about salaries, jobs, loans, and career preparation.

    Thanks, LawProf, LST, and others who have been pushing so hard on these fronts. There's still so much to do, but you've definitely stepped up as leaders.

    ReplyDelete
  14. 4:34, OSU Law has a course on leadership, as well as an extracurricular program on law and leadership. I suspect a number of other law schools do as well. It's a field that I think we should nurture within law schools; among other faults, our current pedagogic structure encourages students to be employee/followers rather than entrepreneur/leaders.

    Related to that is an idea I sent to my dean yesterday--Mayo's Medical School is developing a new MA in the Science of Health Care Delivery, which all MD students in their program will earn *as part of* the regular MD program. I.e., the MA will be embedded in the MD rather than offered as a joint degree that adds coursework.

    As I understand it, the Mayo MA will address the organization and delivery of health care both in specific organizations (i.e., how should I organize my medical practice?) and in society as a whole (how the heck can we better deliver health care to patients?).

    We could do something analogous in law by offering JD students an embedded degree on law practice management--both within individual firms and as a societal matter. There are huge changes coming to law practice, such as the possibility of outside investment in law firms (which will make practice much more entrepreneurial). And the "informatics" (human-computer interactions) that undergird law practice are already quite sophisticated. Programs, or even just a few courses, in this area could help grads obtain and hold onto more sophisticated legal jobs. This might also be a good way for some schools to distinguish themselves. Many academics (and even some practitioners) doubt the role of programs like this, but the Mayo imprimateur is pretty persuasive.

    ReplyDelete
  15. DJM: I doubt OSU teaches leadership.

    Now, it may have a class called Leadership, but that's different from actually teaching people to be capable leaders.

    Leadership is essentially a combination of three traits, good judgment, force of will, and charisma. Lack charisma, and you will never have anyone to lead. Lack force of will, and you will never choose to lead them. Lack good judgment, and you'll fail - you may lead for a time, but you'll be a failed leader.

    I think classes on behavioral economics would be extremely useful, they teach you how people act, what incentives they respond to, etc, super useful for leaders, but that's likely about as close as you can come in a formal academic setting. If you're in one of these classes, ask your professor why Rick Perry has more supporters than Jon Huntsman, and when he can't answer, leave.

    ReplyDelete
  16. Because Jon Huntsman is a product of nepotism, a Mormon and an ambassador of a Democratic president?

    ReplyDelete
  17. BL1Y,

    Leadership is not the same thing as a popularity contest. For example, Law School Transparency was largely a two person operation that did not need to win any elections to perform its function.

    Your requirement of judgment is also circular because you've defined it as something whose existence is a function of whether the goal is accomplished.

    I think the only thing a leader needs is for of will, i.e. not being a pussy.

    ReplyDelete
  18. "Related to that is an idea I sent to my dean yesterday--Mayo's Medical School is developing a new MA in the Science of Health Care Delivery, which all MD students in their program will earn *as part of* the regular MD program. I.e., the MA will be embedded in the MD rather than offered as a joint degree that adds coursework."

    How is Mayo adding an entire MA to the degree, without adding to the amount of coursework required?!

    ReplyDelete
  19. @10:02: I never suggested it was a popularity contest.

    If all that leadership required was force of will, steroid boy would be president. There are plenty of people raging against the machine who aren't leaders, because they can't get anyone to follow them. Being a lone rogue, doing your own thing is perfectly fine, but it's completely different from being a leader.

    Force of will is the ability to rally yourself. Charisma is the ability to rally others. Leadership most definitely requires both.

    As for good judgment being circular, in a way it is. I recognize that sometimes the right choice can lose (I've played black jack), but there's no better sign of good judgment than success - continued success, that is. One right move doesn't mean good judgment, but a history of being right is pretty powerful evidence that you know what you're doing.

    ReplyDelete
  20. I disagree. Leadership is the antithesis of popularity - because the whole point of leadership is to change the status quo, and not to give the populace what they want to hear.

    For example, LawProf is a leader, but he is also hugely unpopular among legal academics.

    As another example LST were leaders, but they are persona non grata in the eyes of law school administrators.

    Also, not all presidents are leaders. George Bush was a leader. Barack Obama is definitely not one.

    ReplyDelete
  21. (I'm not saying GWB was a good leader, just that he was one).

    ReplyDelete
  22. BL1Y, this is one of the funniest examples of bullshitting that I have read in a long time--and I hang out with a lot of bullshitters! What are your qualifications to speak on the top qualities of leadership and whether they can be learned? How many organizations have you led? What was their value and how many people did they employ? How many leadership courses have you taken? Have you looked at any of the materials for leadership courses? The Harvard B School case studies, which we use in our law school course, are pretty good; have you looked at those?

    I know a lot of leaders in business, academics, law, nonprofits, etc, and I led an academic unit pretty successfully (creating OSU's John Glenn School of Public Affairs) before stepping down to care for my seriously ill son. A few leaders take your position (just as a few lawyers believe that you can't teach trial advocacy or negotiation), but the vast majority firmly believe that leadership is like any other skill: People bring different talents to it, but everyone can improve their performance through training. Good judgment, charisma, and force of will are important qualities--as they are for many roles--but clearly are not sufficient.

    IMHO, the problem with many elected officials (and I've worked with them too) is that they *don't* know much about leadership. Talk about dysfunctional teams and decisionmaking . . . .

    ReplyDelete
  23. 10:06, I don't know all the details on the Mayo program, but they probably will count courses toward both the MD and the MA. Law schools already do this with joint degree programs (we count, for example, 12 credits from the MBA program or another MA toward the JD, while the other program counts 12 or more JD credits toward their degree); the Mayo program is just a further step.

    It's possible that Mayo will add a few credits to their MD program (just as law schools vary in exactly how many credits they require to graduate). But in essence they are saying that it is more important for their doctors to learn some of the skills of health care delivery than to take some of the other courses or clinical rotations currently taught in medical school. I hope to find out more about the details--just read an article yesterday by chance.

    ReplyDelete
  24. Has anyone thought about slothful law students being part of legal education's quality problem?

    I can't be the only one appalled at students when they claim to have coasted through class, grabbed an old outline, and try their luck at a final exam. That students like these can still get by in this system certainly suggests massive institutional and structural failures. But if classroom instruction is a waste and they have all this time on their hands, why aren't they doing other constructive things to improve their research and writing, gain client contact through pro bono volunteer projects, or networking with local attorneys and judges?

    While the massive amounts of student loan debt amid a market bloodbath still engender deserved sympathy (I'm a 2011 grad still looking for attorney work, by the way), I can't feel sorry for people blowing off opportunities to learn while in school.

    ReplyDelete
  25. CORRECTION: The phrase "attorney work" in my last paragraph should read "paying attorney work." I am currently doing volunteer memo and brief writing to keep up my skills and prevent gaps from showing up on my resume.

    ReplyDelete
  26. 2011Grad. That's fair, but shouldn't someone like you be deservedly upset that after all of that hard work, all the years and all the tuition money - you're forced to work for free, for the experience? Wouldn't a better world be one where you chose a graduate education that led to a job? For example, had you gone to even, oh, physical therapy school you would be making money right now. Shouldn't we move to a world where all graduate programs disclose honest job placement data so that you can foresee and choose to avoid the situation you're currently in?

    ReplyDelete
  27. I just took a look at OSU's new page. I've also seen the old page and this looks to be an improvement. Still, it paints what looks like pretty decent employment prospects. So either things aren't really aren't so bad or the new improvements are failing to address the the most important things.

    I noticed two things. First, it shows 12 out of 13 unemployed people as "not seeking employment" but with the disclaimer that these 12 people are currently studying for the bar exam. I don't like this practice in employment statistics generally, of considering certain job-less people to not be unemployed because they claim to not be seeking employment. I don't like it here either, I imagine these people probably are or will be soon seeking jobs, and the fact that they don't have them already is also indicative of something. But, its only 12 people. So maybe not such a big deal.

    The big thing is the 51% of salaries reported. 100% response rate, and only 51% reported salary. Moritz reports this information accurately, but doesn't really try to draw any inferences about what this might mean. They then report the median salaries for the 51%.

    I guess its a matter of speculation as to how dramatically these medians would change if the other 49% reported and that's a real problem. On the one hand, you can't demand the schools engage in speculation. On the other hand, you have a pretty good sense that the resultant median salaries reported are not representative of the class.

    ReplyDelete
  28. "I guess its a matter of speculation as to how dramatically these medians would change if the other 49% reported and that's a real problem."

    It's not speculation at all. I am certain that the other 49% would cause the salaries to fall. The 51% sample is a biased and unrepresentative sample that should not be used to represent the group. Moritz knows it's an unrepresentative sample, and that's why their numbers are fraudulent.

    ReplyDelete
  29. 7:50: It doesn't advance the cause of transparency to toss around phrases like "fraudulent" in regard to all data that is less than complete. OSU discloses that 51% of their graduates employed at nine months reported salary data. It then reports that data. That is far from an optimal situation in regard to transparency but it's not "fraud" in either a legal or moral sense. Given that plenty of law schools have committed something that could reasonably be termed fraud, throwing the phrase around in a careless matter just undermines the general credibility of such charges.

    ReplyDelete
  30. That word was carefully chosen and apt. It is fraud to present a biased sample as representative of the population.

    Let me give you an analogy. If I put on my resume that I have a 3.87 GPA, and I disclose that it was based on seven classes, when my true GPA of 3.1 is not presented, then that's fraudulent. I am trying to deceive you into imagining a higher overall GPA than I actually have.

    To remove the fraud, Moritz would also have to state that the salary figures for the 51% do not represent the salary figures for the entire class. But of course they do not do that, because they want students to believe that they do represent the entire class's salaries.

    ReplyDelete
  31. Take this quote from Moritz, "All salaries reported by the Class of 2010 ranged from $53,000 in the 25th percentile to $100,000 in the 75th percentile, with a median salary of $65,000."

    By using terms like "percentiles" and "Class of 2010" they are trying to deceive you into believing that these figures represent everyone. 25th percentile conjurs thoughts of the bottom 25%, not the bottom 25% of 50%. Similarly "Class of 2010" makes you think they're talking about everyone.

    Continuing the analogy, imagine if I put the following on my resume:

    "Grade Point Average for reported courses from the 2010 class year was 3.87." Even if I had earlier defined "reported courses" as the seven classes that I chose, that's still a misleading statement.

    ReplyDelete
  32. P.S. If such techniques are not fraudulent, then you can completely circumvent the fraud rules by (a) using a biased sample and (b) disclosing that your sample was only a portion of the entire population.

    "Smoking has no adverse health effects*[1]

    "[1]Based on 10% of the smoking population."

    ReplyDelete
  33. Your analogy is inapt because you know what your actual grade point average is, and you're intentionally choosing to present an unrepresentative sample of your grades. OSU doesn't know the salaries of 55% of the class. The school doesn't disguise this. It could, I suppose, be even more explicit about that, but a prospective law student who isn't capable of drawing the extremely obvious inference that the data presented is unrepresentative isn't somebody who should be a lawyer.

    ReplyDelete
  34. As demonstrated by University of Chicago, Moritz could have easily gotten the salary data for the other 49%. They chose not to. They designed a process where only those with high salaries would report the information.

    Imagine if a cigarette company does a study of the health effects of smoking, by doing blood work on smokers finishing a marathon. That's a process designed to create a biased sample, just like Moritz's salary reporting process. Then the company reports this as "smokers have 120/80 blood pressure, high oxygen levels [and other statistics demonstrating good health." Like Moritz, they disclose that their study was of a specific group. That wouldn't make the statement any less fraudulent. All it does it make it a very carefully contrived form of fraud.

    If you want to argue that 0Ls should be smart enough to see through this then that's fine, but they're not. Again note all of the verbeage in Moritz's statement designed to get the mind to imagine an entire class:

    "All salaries"

    "the Class of 2010"

    "25th percentile"

    "75th percentile"

    "median salary"

    ReplyDelete
  35. "It could, I suppose, be even more explicit about that, but a prospective law student who isn't capable of drawing the extremely obvious inference that the data presented is unrepresentative isn't somebody who should be a lawyer."

    That's a little callous, Professor.

    ReplyDelete
  36. I might agree with the statement after graduation (i.e. lawyers should "think critically"), but 0Ls are often trusting and naive. The whole reason they go to law school is that they believe in things like fairness and justice.

    ReplyDelete
  37. LawProf is exactly right. The grade point analogy which was floated on another thread is inapposite, for the reason LawProf said. The statement that only a certain number people responded works as a disclaimer. The person reading this should know they don't have enough information to make the decision whether to attend the school. This is not high order reasoning here.

    ReplyDelete
  38. Why doesn't the statement that only a certain number of courses were included in the GPA similarly work as a disclaimer?

    What about the cigarette company study of smokers finishing a marathon?

    Again, if you're allowed to get away with using biased samples by simply disclosing that you only looked at a portion of the population, then I can completely circumvent the fraud rules. Give me any example of what you would consider fraud, and I'll show you how.

    ReplyDelete
  39. Actually students and schools were playing such games with their GPA for decades. See for example the phenomenon of grade inflation.

    That's why legal employers don't care about your GPA. They want to know your class rank.

    ReplyDelete
  40. You are mixing arguments. You are in control of your information. Intent to deceive people can be inferred from your failure to present the info. Schools are not in control of salary information. They do not report at 100 percent because they do not have it. You cannot infer intent from saying," here is what we have, but you should know it is not complete". There is a difference between a bad study and a fraudulent study.

    ReplyDelete
  41. Moritz is in control of their salary reporting. Getting salaries for the other 49% is very easy and doable. They don't want to do it because they would rather mislead.

    ReplyDelete
  42. In the GPA example you are withholding crucial information which you actually have. You know your other grades. OSU doesn't know the other salaries. It isn't picking and choosing which salaries to present, which what is happening in both the GPA example and the cigarette example. Now you could argue that OSU should work harder to collect salary data. Obviously the main reason Chicago has twice as much salary data is that people with good jobs and high salaries are far more likely to report them. But OSU isn't withholding data it has, which is why your analogies are weak.

    There's a serious question here, which is how explicit a school should have to be about the fact that the data it has is unrepresentative. If you want to treat law schools like cigarette companies (which is arguably justifiable) then schools should have to be very explicit about the extent to which the data they have is unrepresentative. But there's a huge difference between simply reporting medians and means without revealing the percentage of graduates these are based on (which is what almost all schools have done until the last year) and doing what Chicago and OSU are doing.

    ReplyDelete
  43. That's where we'll have to agree to disagree then. You say that OSU does not know the information. I say they do, or that they could easily get it. OSU has a world class statistics department, that gathers and analyzes data that is much harder to extract and of much greater volume. Getting salary information for 100 students is a piece of cake.

    We should also not assume that the schools are reporting everything they have. How do we know that someone at career services isn't drawing a line, below which salaries will not be reported even though they were disclosed to the school?

    ReplyDelete
  44. How do they easily get it? What is the process they will use to make people tell them how much they make?

    ReplyDelete
  45. Ask The University of Chicago School of Law what process they used, because they managed to get salaries for 95% of their graduates. Or ask OSU's statistics department to design the process since they can probably tell you what works and what does not.

    But I'm pretty sure a simple phone call to the student, in which you discuss their career search would reveal the information.

    Regardless, if a process is getting you a biased sample then throw it out. Don't be that cigarette company who gets subjects by waiting at the end of a marathon. That's a bad process. Don't use it.

    But again let's not assume that the schools are reporting what they have. I find it very hard to believe that a graduate of Chicago is ten times as likely to disclose his or her salary than a graduate of NYLS.

    ReplyDelete
  46. The respective stats released by the schools indicate that a U of C grad is about four and a half times more likely to report his or her salary than a NYLS grad. That doesn't strike me as implausible at all. (This assumes that almost all of U of C's salary data is self-reported. That is extremely plausible, given that if you look at the graduates of any school who get real jobs at big firms or have good government jobs almost all of them report their salaries).

    ReplyDelete
  47. I may misremember, but I thought 10% of NYLS graduates reported salaries where as 95% of Chicago's reported them.

    Any way, whether it was 10% or 20%, it seems implausible to me and suggests that NYLS rigged the process in some way.

    Here is NYLS's survey
    http://www.nyls.edu/user_files/1/3/4/16/ERSS%20Survey.pdf
    Why would someone fill out all of that information but put nothing in the salary box?

    Take SPECIAL NOTE of the fact that NYLS tells you to fill out the salary box ONLY IF you are "Employed in a position for which you receive a salary or a stipend." What about people working as unpaid interns, working on a day-by-day contract basis (i.e. doc review), or otherwise? They're told NOT TO FILL OUT the salary section. That is a perfect example of designing a process to create a biased sample.

    ReplyDelete
  48. You were the one who said it was easy. You either know it is easy or you do not. If somebody sent me a survey and I deliberately did not answer one or some of the questions, and somebody called me up to try to make me answer, I would be ticked off.

    ReplyDelete
  49. Can you at least speak with the University of Chicago or a statistics department, or make such phone calls before you declare the attainment of salary data for the other 49% an impossible task?

    ReplyDelete
  50. I agree with some of the criticisms of the OSU page, although the critics are overlooking a few important disclosures: We fully disclose that only 70.8% of the class was employed in F/T permanent jobs, that 19.0% of those seeking work held only P/T or temporary jobs, and that only 64.4% of the class held jobs that required bar passage. We also disclose the percentage of students in jobs funded by the law school. These are very important disclosures that LST has worked to get.

    Also note that, in addition to making disclosures in all of those categories, we report stats as a percentage of *both* grads seeking work and total grads in the class. This means we don't play games with where the "non seeking" grads are allocated. E.g., the figure would look best for P/T or temporary jobs if we based it on the full class; conversely, the figure for F/T permanent looks better when based just on those seeking. We disclose both for all categories, rather than playing games with that. We report why some grads weren't seeking jobs (which is sort of damning in itself, since some were taking the bar for the second time), but don't exclude them from the analysis.

    I am working to improve the salary data in two ways. One is to collect my own data from the class of 2010; the other is to develop a way of adjusting the medians given the missing data. (Social scientists work all the time with missing data--51% is actually a pretty good response rate in social science--and there are ways to deal with missing data.) After I've done the latter (which I can do with current info), I may be able to get the deans' office to change the way this is worded. If not, I'll give the info/approach to LST (well, I'll do that anyway, so it can be applied to other schools) to make the information more useful. You can actually deduce a lot about salary from what's reported here, but I agree that 0Ls shouldn't have to do it on their own.

    I'm so glad I won't have to give up nagging for the new year!

    ReplyDelete
  51. Do you have a copy of OSU's graduate survey DJM? I'd like to see if they're playing any of the survey games that NYLS uses above.

    ReplyDelete
  52. "51% is actually a pretty good response rate"

    If true, that makes University of Chicago's 94% response rate that much more amazing. I guess some schools deserve their high ranking. Everything they do is of the highest quality.

    ReplyDelete
  53. @10:51--I don't know who you are addressing, but you can call Chicago as easily as anyone else. They sound as if they would be happy to talk with folks about what they do.

    ReplyDelete
  54. DJM,

    Is there any way you, or Bill Henderson, could get the survey forms from career services and audit the reported data? I don't necessarily mean that you would audit them by independently contacting the graduates and verifying the information they wrote. That would be nice, but I mean a less intensive audit. I mean that you would simply confirm that the information reported by OSU matches what was in the surveys.

    ReplyDelete
  55. DJM it sounds as if you have given 0Ls what they need to make a decision whether to go to your school.

    ReplyDelete
  56. OSU's response rate among 2010 graduates being surveyed regarding their employment status wasn't 51%: it was 100% (this is actually higher than Chicago, which was unable to determine the status of one graduate in its 2010 class).

    The problem is that the school got a response rate of 51% among employed graduates *to a specific question* i.e., salary (this number represents 45% of all graduates in the class). Chicago appears to have gotten close to a 100% response rate to the same question, although the school says it didn't rely only on graduate responses when compiling salary data.

    ReplyDelete
  57. Interesing, if Chicago didn't rely only on graduate responses, what other techniques did they use?

    ReplyDelete
  58. "Employment and salary information is taken from a variety of sources, including: student self-reporting, alumni surveys, employer information, published salary data, and electronic resources."

    ReplyDelete
  59. Has anyone reading these comments filled out a survey by (a) putting themselves in the full time employed category and (b) putting nothing in for salary?

    That just seems so odd to me. I can see not filling out the survey at all. But I don't understand why anyone would fill out all but one question.

    ReplyDelete
  60. That is interesting. What "employer information" would be different than "published salary data"? And what do you mean by "electronic resources"?

    ReplyDelete
  61. I can see it easily. You don't mind telling people that you are working. But you don't give them salary information. That does not seem unusual to me at all.

    ReplyDelete
  62. There are some curious although probably insignificant inconsistencies across Chicago's data. For example, they report salaries for 184 of 204 graduates in the class of 2009, which is a slightly lower percentage (90.2%) of graduates than the graduates they reported having salary data for when they reported that class's numbers to US News, at least per LST, which lists 92.4% of the 2009 class's salaries as known (LST uses USNWR's numbers).

    ReplyDelete
  63. Here is what I know in response to some of the questions:

    As far as I know, all schools use the same form to collect employment and salary data; it is the form that NALP creates. Now that the ABA is mandating disclosure, I assume the ABA will create a form--that may take the place of the NALP one.

    People--in any occupational field--do not like to disclose their individual salaries in a way that can be connected with their names. This is well known in the social science literature; even the government has trouble getting people to report salaries voluntarily. (Again, that's why BLS relies upon payroll records rather than self reports of salary.) People are even less likely to tell someone their salary over the phone. If it's the top salary in the field, sure, there's a good chance that an individual will disclose it. But if it's less than the top, it's hard to get disclosure.

    If you talked to data collectors in a sociology or economics department (statistics folks analyze; they rarely collect), they would counsel the same approach that I think LST has advocated--collect salary data anonymously. This means that the collection of salary data has to be separated from some of the other data collection. You can collect both types of information--and be more successful in collecting salary data--but you have to separate the inquiries. The NALP form ties all data to the individual student, and that's a big part of the problem with collecting the salary data.

    There are two keys to Chicago's high rate of salary reporting. One is that a large percentage of their students have the top starting salary in the field, so they are more likely to disclose their salaries. The other is that most of the Chicago salaries are publicly available--law schools can rely upon Above the Law or any other source for salary reports. Schools can also apply one student's reported salary to another student with the same position. So if Chicago has 5 students who go to Winston & Strawn, and only one fills in the salary space, then Chicago can attribute that salary to the other four--it's a reasonable assumption, at the entry level, that all 5 are earning the same salary. Chicago can also call up one of its alums at W&S and ask "what is your entry level salary this year?" The alum will respond, and that covers 5 salaries with one call.

    It's not impossible to get good salary data at schools outside T6, but it requires a different sort of work than the approaches Chicago takes. And in the end, more data will remain missing at schools outside T6. But that's not the end of the road--there are much better ways to interpolate those data than anyone is using now. Basically, schools aren't using any type of interpolation and that's part of the problem.

    ReplyDelete
  64. "I can see it easily. You don't mind telling people that you are working. But you don't give them salary information. That does not seem unusual to me at all."

    It does to me. I don't know you would assent to filling out the employment survey, but then when you see that one question about salary, you would refuse to fill it out. What benefit is there to hiding that information from the career services office?

    I would bet that students are reporting this information, and that career services is hiding it.

    ReplyDelete
  65. It does not take such a granular approach to know that it makes sense to go to the University of Chicago. There is a whole history beyond one year's statistics.

    ReplyDelete
  66. DJM,

    The problem with your claim that people do not like to report salaries, except when they make a lot of money, is that Chicago has plenty of low wage earners (clerks, small firms, government . . .) reporting salaries.

    Regarding the need to have anonymous salary reporting, when you report salaries on employment surveys it is anonymous. They are not going to disclose your name and salary to the world.

    I think there's something far more nefarious going on at lower ranked schools to hide the salary, which I would call a combination of (a) designing processes that inhibit the reporting of salaries and (b) not reporting salaries that were disclosed to the school, because they make the school's placement rates seem bad.

    ReplyDelete
  67. "It does not take such a granular approach to know that it makes sense to go to the University of Chicago."

    I disagree. Someone choosing to go to U of C is not making that choice in lieu of going to Kent School of Law. They are choosing between U of C and NYU, or they are choosing between U of C full sticker price and Kent with full scholarship. These types of decisions require solid data.

    There is a big difference between a school where 98% of graduates get a good job, and one where 75% get a good job (which I suspect is what happens at NYU).

    ReplyDelete
  68. DJM, how much knowledge do you have of how salary info is gathered by career services offices? I myself don't know how this is done. I'm struck by the extent to which the percentage of salary data reported to NALP is very tightly correlated with the size of the firms for which graduates work. For example in the 2009 data, salaries are reported for 35% of graduates working for firms of between 2-10 lawyers, 60% of graduates working for firms of between 11-25 lawyers, 68% of graduates working for firms of 26-50 lawyers, 82% for graduates working for firms of 51-100 lawyers, 88% if graduates working for firms of 101-250 lawyers, 92% of graduates working for firms of 251-500 lawyers, and 95% of graduates working for firms of more than 500 lawyers.

    As you say people with high salaries are far more likely to report them than people with low salaries, so that no doubt explains much of these differences. But I'm wondering how much extrapolation law schools do based on public data. Under current conditions that could be dangerous, as an increasing number of graduates are working for big firms, but as explicitly temporary hires -- and not just in doc review, but more generally.

    ReplyDelete
  69. 12:21, the salaries of judicial law clerks and most government employees are completely public--the numbers are easy to fill in for Chicago's grads in those positions. Chicago has a relatively small percentage of 2010 grads working in firms of less than 50 lawyers (less than 10%)--although it's striking how that percentage increased in 2010. That change in itself says something about the shifting job market.

    There is lots of deceptive reporting, and probably even some nefarious practices (if it happens with admissions statistics, it would be naive to think it couldn't happen with placement). But the bulk of the problem is different from what most of the commenters here are focusing on--and you won't solve the problem if you're not willing to consider other means of attack. Gotta go for the rest of today--need to finish compiling materials for the Business of Law seminar and work on my own 2010 survey.

    ReplyDelete
  70. One way to read the numbers you quote is to conclude that people with high salaries are more likely to report the data. But another interpretation would be that students from good schools are more likely to report the data, because their administrations don't try to prevent such reporting.

    The way to answer this question is to see the reporting rates, broken down by the rank of the school.

    For example, I would bet that the percentage of graduates in firms of 2-10 lawyers who report salaries is strikingly different for graduates of a top-5 school, than it is for graduates of a tier 2 school, because the top-5 school is not trying to hide the data.

    ReplyDelete
  71. Number of U of C graduates joining firms of 2 to 10 people: 2008: 1 2009: 1 2010: 6

    ReplyDelete
  72. It reminds me a bit of the parable of the ant and the grasshopper.

    The grasshoppers were the people who coasted through law school and hardly ever studied except to look at a commercial outline a few days before the exam.

    The ants are those who worked hard, engaged with their professors, and built up good grades and connections.

    When the weather (and job market) was good, the grasshoppers enjoyed themselves and laughed at the ants and all the hoops they were jumping through.

    But now the cold winter of a weak job market is upon us.

    ReplyDelete
  73. Regarding salary data, the folks running IBR have it for people in IBR. You have to send them your tax return every year or they kick you out of IBR.

    It'll be interesting to see how the government uses that data.

    ReplyDelete
  74. @12:3O. NYU is a great school with lots to commend it. I could see someone choosing to be in NYC over Chicago. The big difference you see is not the sole motivating factor for everyone.

    ReplyDelete
  75. One more quick comment, since I just saw LawProf's direct Q for me: I think a fair amount of the salary reporting (I'd guess at least 25%?) is inferential, although I don't know precise percentages. Another point that hasn't been noted yet is that students often don't fill out the forms themselves--the career services office is allowed to do that for them. If a student tells the career services office orally "I got a job as an associate at Jones, Day in Cleveland!", the career services office can fill out the form with that info and put in the salary number (which they undoubtedly know from other sources). Same thing happens when a grad orally reports "I got a job with Merritt and associates," except that the career services office doesn't have a clue what Merritt might be paying. So the form gets filled in with job title and employer, but no salary.

    I suspect this problem becomes even more marked with jobs offered after graduation (and an increasing percentage of jobs outside T6 fall in that category). There's not much incentive to fill out that long boring NALP form at any time--and schools have no leverage once the student has graduated. But the grad may email career services--or even one of his professors--to celebrate getting job X at employer Y. As I understand it, and I'm pretty sure this is correct, that info can be recorded on the grad's NALP form by the career services office.

    There are a lot of ways this could be improved. E.g., phone calls to grads probably would elicit *some* additional salary information--it just wouldn't be the near-perfect Chicago score. So would polite emails, especially if couched in terms like "we really want to help our graduates and prospective students understand what salaries they might expect; please help us convey as accurate a picture as possible. We will keep your personal information completely confidential."

    But I think the percentage you calculated, LawProf, offer a clue to one way to wring more accurate data from the current numbers. As you note, the percentages reporting go up as the size of the firm increases. But the number of grads employed in those firms of different sizes goes down. I think part of what we/I have to do, is use the median salary reported by grads in the 2-10 firms for *all* grads in that group, then do the same for each other category. This gets a little tricky mathematically, but I think there's something to be done here. It's sort of the next step from what LST did with the US News numbers.

    But I'm feeling guilty that I'll hold up the copy room operator, so back to those course materials for sure.

    ReplyDelete
  76. I believe that it should also be reported in the interest of full disclosure that for those graduates taking jobs in AmLaw 200 firms and reporting starting salaries of 160K, according to a Citi Accounting report from Q3 2010, 80% will not be employed by that same AmLaw200 firm 5 years later (based on 2010 data but also reflective of past performance as well). Some of that 80% will lateral to another big firm or smaller firm, some will go inhouse to a corporation, some will go to government and some will not work in another lawyer job.

    ReplyDelete
  77. I do not think that every school should have to report that general information every year. Just as you found it, others can too. This is a statement about the profession overall. It is, probably, a relatively constant number. It is not too much to ask that prospective students gather general unformation about the profession they want to enter. The school-specific stuff is another matter.

    ReplyDelete
  78. Here are two (partial) solutions for the "no responce" conundrum on graduate salary surveys. One, if a school only gets a 10.2% responce from its salary questionnaire it would be required to start its salary discloture with the statement, "Only 10.2% of our 2010 graduates report having any income from the practice of law." How long would it take before the schools acheived dramatically higher responce rates? Or two, why not report the percentage of graduates who are paying their law school loans on the original terms (i.e. not counting IBR, deferments and such as payment)? If only 40% of the loans were being paid as agreed then the law school discloture would state, "60% 0f our class of 2010 have defaulted on their law school loan payments." That puts the matter starkly to 0L's. William Ockham

    ReplyDelete
  79. "Only 10.2% of our 2010 graduates report having any income from the practice of law."

    Well said. A biased sample can be presented in a way that misleadingly makes the school look good, or in a way that makes them look really bad. I like this solution.

    ReplyDelete
  80. I am a 3L student who have been using the "coasting" method since 1L year. I work two part-time jobs to try and get out ahead of my soon to be humongous debt payments, volunteer at the Attorney General's office, and also work on research projects for the law school clinic, a professor. I hold an executive board position in a journal and am leader of another club (as both I am trying to increase student interaction with alumni). Last year my schedule was similarly demanding. I would challenge anyone to tell me how I would get more "value" out of spending my days in class, doing the reading, and outlining.

    I sincerely wish I could simply pay the 50K in tuition with a gentleman's understanding that I would get a guaranteed B average so I could devote more time to outside work.

    Do I think my classmates who are still "coasting" but aren't doing as much work are doing it wrong. No. First, I know very few coasters who are not interning or working somewhere part time, even if it is 15 hours per week as opposed to the 40 I spend working/interning. Second, I wouldn't discount the value of self-study. I wish I had more time to read on my own. I learn a lot more by reading a series of interesting books about a topic than I do by taking classes. I wouldn't even discount the value of socializing. I can see how devoting extra time to socializing during upper-years can lead to some powerful relationships down the line.

    ReplyDelete
  81. Am I the only one who's perplexed by "nyc" and his bizarre obsession with "jumping through hoops"?

    ReplyDelete
  82. I'm pretty sure nyc is a very high level troll (meaning a troll so subtle that you can't accuse him of trolling, even though he is).

    ReplyDelete

Note: Only a member of this blog may post a comment.