The impact of scholarship on teaching -- not much!

I've just encountered a very nicely executed and insightful study by Ben Barton of the University of Tennessee College of Law, on the question: "Is There a Correlation between Law Professor Publication Counts, Law Review Citation Counts, and Teaching Evaluations? An Empirical Study." It's available for download on SSRN and was published in 2008 in the Journal of Empirical Studies.

Barton studied the scholarly productivity and impact, and the teaching evaluations received from students, for every tenured or tenure track faculty member at a diverse group of 19 law schools in the United States -- 623 faculty members in total. I'm no statistician, but my impression is that his statistical analysis is done with care, and with a recognition of the many imperfections of all the data -- which nonetheless remain the data that we have. The result is a finding that "there is either no correlation between teaching evaluations and these measures of scholarly output, or a very slight positive correlation" (16). Moreover, this finding appears to be quite consistent with the results of studies elsewhere in higher education (which Barton summarizes at 2-3), and his data appear to cast doubt on one much smaller study of law school teachers that had found a greater positive impact (described at 3; for Barton's contrasting analysis, see 18).

What should we make of this finding? Barton points out that his results are inconsistent with two quite opposite hypotheses, each with its adherents. (19) One group presumed that the impact of scholarship on teaching would be positive, on the theory that it is through scholarly work that teachers master their subject. But the impact was at most slight. The other group believed that the impact of scholarship on teaching would be negative, because the time required for doing scholarship would inevitably take away from the time a professor could devote to improving his or her teaching. But this effect also turns out to be absent.

This pair of results is actually quite odd. Scholars do learn about their subject as they write about it, or at least they feel they do (I personally feel I do) and it would make sense that they would. But this increased learning has little impact on their teaching. At the same time, scholarship takes time, and time is scarce, yet this substantial claim on scholars' time doesn't turn out to demonstrably impair scholars' teaching.

One possibility is that both hypotheses are right, and that they are mostly invisible in the data because they cancel each other out. That is, scholarship does enhance scholars' knowledge, and it does take away time from their work on their teaching -- and so what is gained on the one hand is mostly lost on the other, with the net result (this is what Barton's data say) that productive scholars are at most only slightly better teachers than their nonproductive colleagues.

But what if actually both hypotheses are wrong? In that case the reason that scholarship has little impact on teaching would be that (a) scholars don't learn that much from their scholarly work that can help them in their teaching and (b) the time scholars spend on their scholarship doesn't much impair their efforts to be good teachers. So again the two effects, or rather non-effects, balance each other out. Could these two propositions be correct?

As to the first, it might be argued that although scholars do learn about their subjects as they write about them, they don't learn much that they would want to convey to their students. If most scholars today are engaged in various forms of esoteric theory, then it might indeed be the case that while they learn a lot from their writing, what they learn is not what they teach. In fact, Barton finds some evidence that "practice-oriented scholarship" has the greatest impact on teaching evaluations (15) -- an ironic result, since the kinds of scholarship Barton quite reasonably appraises (see 8) as the most practice-oriented (treatises, casebooks, and "practitioner article[s] or chapter[s]") are probably not those viewed as most prestigious among scholars today.

But how could it be that the time spent on scholarship -- if it's not a positive benefit to teaching -- doesn't wind up actually impairing teaching by taking time away from scholars' focus on it? Two possibilities immediately suggest themselves. One is that the tenured and tenure-track people who don't do scholarship also don't spend much time on their teaching -- so the scholars are as attentive to their teaching as the nonscholars. The other, a much happier possibility, is that although the scholars spend less time on their teaching than they otherwise could, they (and their less productive colleagues) still spend enough time to do a good job. It may well be that tenured and tenure-track law faculty -- busy as they may feel at times -- in fact have so much time to devote to their teaching, even after they finish their research, that they can and do prepare themselves well for teaching.

I'm inclined, however, to reject all of these explanations, or rather to say that they are all unproven. I wonder if what we are seeing is a different phenomenon at work. There surely are better and worse law teachers, but I'm inclined to think we do not yet know much about how to describe who the better and worse teachers are, or about how to convey to less effective teachers the skills that will make them better. The result, I suspect, is that we cannot really measure the impact of scholarly work on teaching, because we are still at such an early stage in developing ways to improve our teaching.

One last point is important to make. Barton studied only tenured and tenure-track law faculty, because (at least usually) only they are expected, as part of their jobs, to produce scholarship. We know even less, therefore, about the impact of scholarship on the teaching of those faculty who aren't required to write, but choose to do so nonetheless. We also don't know very much about the impact of scholarship on the teaching of those faculty whose principal teaching responsibilities are in "lawyering skills" rather than in legal doctrine, since skills teachers are probably still much less likely to be tenured or on tenure-track than their doctrinal colleagues.

In short, there's a lot we don't know. I wouldn't take Barton's study as demonstrating that scholarship is without value to teaching -- though it does demonstrate that scholarship has little demonstrable impact on teaching. Instead, I would urge that we focus most directly on what seem to me to be the central uncertainties: how to be, and how to help others to become, better teachers. If we can work on these issues, I think we can safely put to one side for now (and probably for the foreseeable future) the seeming tensions between scholarship and teaching.