Saturday, March 19, 2011

Do high LSAT scores and high college grades help -- or hurt -- lawyers?

In my previous post I tried to comment on some overall implications of the Shultz-Zedeck study. But several of the specific findings just cry out for discussion as well, on the complex relationships between measures of academic ability or achievement on the one hand and elements of lawyering effectiveness on the other.

Shultz and Zedeck generated an enormous quantity of data, including appraisals of study participants' competency on 26 different indices, as measured by the participants themselves, or by their peers, or by supervisors, or by peers and supervisors together, or by all evaluators together. In evaluating the extent to which various possible predictors in fact correlated with the subjects' lawyering competency, they focused (and I do too) on those instances where a predictor (such as the LSAT) correlated with a competency as appraised by at least two groups at a statistically significant level.

In the part of their study that focused on current Boalt law students, Shultz and Zedeck found that those with higher LSAT scores did better on a number of effectiveness measures, including not only "Analysis and Reasoning" -- presumably the principal focus of the LSAT itself -- but also "Writing," "Influence and Advocating," "Creativity" and "Problem Solving." (73) This finding is actually quite a comforting one for schools relying heavily on the LSAT in their admissions decisions, or in other words for almost all American law schools. It suggests that the abilities the LSAT measures are not just those of technical legal analysis but also include some more supple skills that surely are very much needed in actual practice. On one criterion, however, these students did worse, namely "Networking," and Shultz and Zedeck wonder whether high-LSAT students tend to be those who aren't as good at Networking or give it less attention. (73)

Results for the alumni/ae study subjects, who included graduates of both Boalt and Hastings, were broadly similar (53-54). Higher LSAT scores correlated positively with "Analysis and Reasoning," "Researching the Law," and "Writing"; and negatively with "Networking" and "Community Service."

Meanwhile, Boalt students with higher undergraduate grade point averages (UGPA) turn out to score lower on "Practical Judgment," "Questioning and Interviewing," "Developing Relationships," and "Community Service." All of those, the authors suggest, might reflect that those with higher grades are more focused on their books and less on engaging with others in the world. (73-74) Why wouldn't these same effects appear with higher LSAT scores? Perhaps the answer is that the LSAT tends to test just ability, while the UGPA reflects application -- and application to one's books takes time and energy that otherwise might be devoted to learning how to engage with others.

Interestingly, the same results didn't appear with the alumni/ae studied. For them, higher UGPAs didn't correlate at a statistically significant level on any of the competencies where Boalt students with higher UGPAs seemed to be weaker. (See Table 18.) Instead, among graduates, higher UGPAs correlated with higher competency in "Writing," "Managing One's Own Work," and "Diligence." (54) Perhaps the diligent people who earn high grades, and show some gaps in interpersonal competencies in their student years, apply themselves to their interaction with others once they are in practice and over time make up for their bookish focus as students. Perhaps those with high LSATs, who seem to be less skilled at Networking whether assessed as students or as graduates, just never do become interested enough in this task to master it.

The most dismaying feature of the impact of higher UGPA among the students is that it also results in lower scores on "Integrity." (73) All appraisals of Integrity -- by the students themselves, by their peers, by their supervisors, by their peers-combined-with-supervisors, and by everyone combined -- correlated negatively with UGPA, though the correlations for the appraisals by the students themselves, and by their supervisors alone, were not statistically significant. (Table 48.)

Shultz and Zedeck don't comment specifically on why this might be so. The bleakest explanation would be that those with the higher undergraduate grades sometimes didn't come by them honestly, and this lack of integrity shows up in their lawyering behavior as well. But I wouldn't read the finding this way, nor as an indication that people who study are in some way less fundamentally concerned with morality than their peers. What I suspect it indicates is that "Integrity" as a competency is a quality of action, and that those who have spent more time studying and less time in action have had less opportunity to develop themselves as moral beings. This might also explain why there's no statistically significant relationship between Integrity and students' LSAT scores; though students with higher LSATs do show signs of inattention to others in their lower "Networking" scores, they may be less inattentive to others than those who are putting in the work to earn higher UGPAs.

Interestingly, in the study of alumni/ae (at 54), higher LSAT scores did correlate negatively with self-appraisals of "Integrity," but positively with appraisals of Integrity by others (specifically, with appraisals by peers, and by peers-combined-with-supervisors, but not with appraisals by supervisors alone, see Table 18)). Shultz and Zedeck don't discuss it, but their Table 18 reflects the same kind of interaction between "Integrity" and the UGPA among the alumni/ae: a negative correlation between grades and Integrity as measured by self-appraisal, and a positive one for Integrity as measured by "Other" (that is, by peers and supervisors combined).

Shultz and Zedeck do address the LSAT - Integrity relationship, and suggest that the explanation may be that the "Self" ratings reflected participants' knowledge of their own secrets, while appraisals by others reflected the impact of successful manipulation of appearances by the guilty "Selves." (54) Presumably the same reasoning could explain the UGPA - Integrity relationship. And perhaps the reason that Integrity as appraised by all raters in the student study correlated negatively with UGPA, while Integrity correlates negatively with UGPA (and LSAT) for alumni/ae only when Integrity is appraised by the graduates themselves, is precisely that over time people learn to cover up their moral flaws from others.

But I prefer a different explanation, one that takes seriously the idea that integrity is an interpersonal quality. Over time, even those with relatively weaker interpersonal skills can learn what integrity requires -- so alumni/ae with higher UGPAs or LSATs are not in fact lacking in integrity in action. But if an inattention to others underlay their higher UGPAs and LSAT scores, perhaps that reflected a greater preoccupation with attention to themselves, and perhaps that inner-directed focus tends throughout their lives to make such people more self-critical than their peers are.

The main finding, however, is probably the one Shultz and Zedeck emphasize, namely that neither the LSAT nor UGPA nor an Index combining the two predicted much about the 26 elements of lawyering competence among alumni/ae (55) or among students (74). I don't think it follows from this overall point that intellectual ability and achievement are unnecessary to lawyering competence. Rather, I think what the finding underlines is how multifaceted good lawyering is, and how many different human attributes need to be nurtured to help people become effective lawyers.

No comments:

Post a Comment