Dear Professor Hattie (Or Other Visible Learning Expert)
Let me make this clear at the outset: I think many of the ideas and findings in Visible Learning are useful and encouraging. I am a little concerned, though, about a few things. Especially since I am suspicious about the unencumbered zeal with which school administrators and many teachers are treating your findings. (I am always suspicious of any kind of unencumbered zeal.)
Please help me out with the following questions. (And please excuse me if the answers I need are readily available – I can't seem to find much online: most of what I have found is either about your results or in the form of a critique of your statistical methods.)
- How do the studies you include in your meta-study define the terms 'learning' and 'teaching' exactly? (Is 'learning' defined as the ability to access facts and 'teaching' as the art of transferring those facts? Or could they be something else? Do all of the studies define teaching and learning in the same way? If not, do you think it's fair to group them together? For example, if teaching is defined as preparing students to be creative problem-solvers, and it is found that class size does not play a role in this kind of teaching and learning, are the results lumped together with studies which base learning on the results of a standardized test?)
- Were all of the studies you included in your meta-study of the same quality? (That is, were they all subject to similar levels of rigorous peer review?) Also, am I correct that some of the studies you included date back to the 1980s? And if so, do you give these as much credence as more recent research?
- What was the variation in sample size, and did you weigh each of these equally? (That is, was a study using only 50 students given the same weighting as those involving 500 or more?)
- Am I correct that your meta-study melds together findings regarding learners from ages four to twenty (kindergarten to college age) as one single group? (Do you see any problems with this approach?)
- Have you looked at all at how some factors influence other factors – either to bring about an enhanced combined effect or to cancel one another out?