The “2010 Survey of Law School Experiential Learning Opportunities and Benefits,” just published by NALP (The Association for Legal Career Professionals) and the NALP Foundation, provides striking evidence that law graduates – or at least associates at firms almost all of which had 100 lawyers or more – regarded their clinic and externship experiences as valuable in preparing them for the practice of law is important. But like most surveys it raises at least as many questions as it answers. Here are some that struck me, based on what I’ve digested so far from the survey report (full disclosure: my colleague Meg Reuter, Assistant Dean for Career Planning at New York Law School, was a vice-chair of the NALP work group that helped develop this study):
First, as important as the core finding about the value of clinics and externships is, there is more that would be good to know about it. The survey asked students to list all the sorts of practice courses they had taken, and the report provides the ratings of “usefulness” of various sorts of courses and experiences – clinics, externships, (classroom) skills courses, and pro bono work – by the students who took those particular kinds of courses. It’s very interesting to know that students who took clinics rated them as valuable, and students who took externships rated those just about equally highly (and that students who took simulation courses were less impressed, and students who did pro bono work even less so). But it would also be interesting to know – and it seems the survey data could tell us – how students who had more than one of these courses or experiences rated them: how, for example, students who took clinics and simulation courses rated those two kinds of classes, or students who took clinics and externships. Because they have a basis for comparison, the judgments these students make may be worth careful study. It would also be revealing to know more about which clinic experiences and which externship experiences correlate with high assessments of usefulness; the survey collected some data on types of clinic and externship experiences and so perhaps these correlations can be reported and studied, but they’re not in the current report. (Rebecca Sandefur and Jeffrey Selbin have noted the importance of breaking out the various different sorts of clinics in their analysis of another study of preparation for practice, the "After the J.D." study, in their article The Clinic Effect, 16 Clinical Law Review 57, 84 (2009).)
Second, although the report demonstrates a striking difference between clinics and externships on the one hand, and other skills courses on the other, the difference needs to be carefully assessed. It’s striking that much higher percentages of respondents considered clinics or externships very useful (63.1 % and 60.1 %) than said the same about classroom skills courses (38.5 %). But these figures by themselves may overstate the real differences between the graduates’ perceptions of these courses. The survey asked for answers on a 1 to 4 scale, with 1 being “not at all useful” and 4 “very useful.” The total percentage giving clinics a 3 or a 4 was 84.3 %; for externships, 83.5 %; and for classroom skills courses, 76 % -- many graduates who’d taken classroom skills courses gave them 3’s and so narrowed the cumulative gap between the various types of courses. According to the report, the “average usefulness rating” for clinics was a 3.4; for externships the same; and for skills courses not much less, 3.1.
On the general question of how to deliver effective skills training, it’s also noteworthy that about twice as many students took one of the classroom skills courses as took clinics or externships (637 in skills courses; 279 in clinics; 333 in externships). So it might be said that the classroom courses delivered “pretty useful” instruction to a much larger number of students than either one of the two types of more experiential courses did. One other point to consider here: the classroom skills courses covered a wide range, including trial advocacy, advanced drafting, “subject matter specific skills,” law practice management, leadership, and “other.” Of those that students took most, all except advanced drafting may fall in the general category of simulation skills training, but it’s not entirely certain which particular classes the students found more or less useful.
One other point about the general effectiveness of skills training in law school: how much difference does it make to those who hire the new graduates? Another commenter has asked whether there are data on whether law firm hiring partners take such courses into account. That I don’t know. The NALP survey asked a related question of the graduates -- which, if any, of their skills courses they discussed in their hiring interviews -- but this report does not include data on what the answers were. It would be heartening to learn that the programs graduates regard as having helped them prepare for practice were in some tangible way being taken into account by those who hired these graduates for their practice jobs – but this happy possibility may turn out not to be the reality of the matter.
Third, the survey seems to have asked only about skills courses and pro bono work. If that’s right, that means it didn’t include some law school experiences that may be very useful but don’t take place in courses. Moot court is the clearest possibility, but many students may find law journal work useful (because it hones their writing and research skills, or because it, like moot court, gives them experience in running an organization). It would be useful to know more about how graduates evaluate these sorts of law school experiences once they’re out in the world. It would also be useful to know whether students see any of the doctrinal courses they’ve taken as useful too. Right now we know that students consider clinics and externships very useful; we don’t know from this survey whether they find their civil procedure or labor law classes very useful too. (We do have data on this from the “After the J.D.” study; as summarized by Rebecca Sandefur and Jeff Selbin, in The Clinic Effect, the new lawyers who were that study’s respondents rated their “upper-year lecture courses,” “course concentrations,” and “first-year curriculum,” as significantly less “helpful … in making the transition to [their] early work assignments as a lawyer” than summer and part-time work, clinics, legal writing training and internships. (83-85))
Fourth, this was a survey of “law school experiential learning opportunities.” As such, it didn’t ask about non-law school experiential learning opportunities. As Rebecca Sandefur and Jeff Selbin have found from the data generated in the “After the J.D.” study some years ago, new lawyers rated their full-time summer jobs as significantly more valuable than any other experience rated in helping prepare them for the transition to practice, and rated part-time jobs during law school at the top of a category of the next most valuable experiences, which included these jobs, clinics, legal writing training, and externships. (85-86) (There is room for some doubt, however, about whether all the raters had actually had the experiences they rate (see 85), and so it’s not clear to what extent these ratings reflect judgments by people who actually had a personal basis for comparison.) But the possibility exists that what our students believe they learn most from is simply real experience. Clinics and externships provide that. I believe some other law school courses, such as project-based learning courses in which students, for example, create a website on a real-world issue, do so as well. But jobs provide real experience too.
If additional data turn out to confirm the “After the J.D.” finding that graduates value their summer jobs as much as, or more than, they value their clinics (or other law school skills instruction), would that mean that the academic instruction in clinics and elsewhere didn’t add anything to the value of experience all by itself? I think it’s quite possible the data will turn out this way, but that drawing that conclusion from them would be extremely implausible. Taking these imagined data at face value, what they’d show is that these two forms of learning are both very useful, not that the same things are learned in each setting. One might ask, also, whether these data really should be taken at face value. It’s entirely possible, after all, that graduates don’t realize all that they’ve learned in any given setting. If it turned out, for instance, that they didn’t consider their first-year courses very useful (as the “After the J.D.” data indicated), we might respond that they were mistaken, because they had to get the foundation in legal analysis and knowledge that the first year provided before they could go on to anything else.
But I think we should recognize that learners do know something about where they are learning most. If they turn out to value both jobs and law school skills training, that is a good reason to consider seriously the value of real experience as part of the overall law school learning process. It’s not, however, likely to be an argument against clinics or against skills training in law schools overall. If it turns out that law school skills experiences and jobs are both very useful forms of preparation for practice, the next question would be whether both are more useful than some or many of the doctrinal courses offered in law school. If the answer to that question turns out to be “yes” – and this is what the “After the J.D.” respondents indicate, though as I’ve just said their judgments may have been mistaken – then the implication would be that both of these forms of experiential training should be accentuated, not that one or the other should be sacrificed while the rest of the curriculum remains unchanged.
So the survey is very helpful. The question of what new lawyers believe was useful in their law school years in preparing them for law practice is an important one, and it would be good to ask it as to everything the new lawyers did during (and in the summers of) law school. Right now we have a part, a tantalizing part, of the whole story.
No comments:
Post a Comment