Why has educational research failed to provide us with any widely accepted sense that we have discovered new and improved ways of educating our children?
It’s Difficult!
There is no shortage of research being undertaken to help try to improve the quality of education provided in schools across the world. By now, we should really have discovered rather more about the best ways for teachers to teach and learners to learn, but somehow there appears to be more disagreement than agreement.
Part of the problem lies with the fact that reliable educational research is extremely difficult to achieve.
This is due to the very high numbers of variables involved and unavoidable levels of bias, in terms of the selection and set-up of experiments, and of the manipulation of subjective data and the communication of results, especially where an agency is keen to prove their hypothesis is the correct one.
Think Tanks:
In the UK there are a number of independent policy ‘Think Tank’ organisations that regularly currently conduct educational research. Amongst others, the most well-known are probably the Policy Exchange, Sutton Trust, Demos and the more recently formed Education Foundation.
Each has its own agenda and political leanings, despite their claims to be independent. At the same time there are also numerous university-based research projects and individuals engaged in MA and PhD research which are expected to have impact.
The main problem however is that most teachers just don’t have the time to engage with the reports they produce. Until policy and/or schools address this, it is left to the motivation of the individual.
Image: Shutterstock
Sharing Research:
Meanwhile ‘ResearchEd’, sponsored by the Education Development Trust and run by @TomBennett71, appears to be the current buzzword in schools at present. It has the noble ambition of getting teachers to become involved in undertaking research activities as part of their everyday experience in the classroom, and sharing the results with others.
Over the past years I’ve been fortunate to be involved with various educational research projects of one sort or another. Each one has proved to be a fascinating and rewarding experience that I would strongly recommend to any teacher. And that’s something I’ll return to later …
But during that time I have developed a healthy disregard for the so-called results, or ‘evidence’ as it is referred to, of much educational research. The chances are that the experiment just isn’t reliably repeatable because the variables – students, teachers, school, content, timing and intentions – are just too variable.
Then there is the problem of interpretation – there have been occasions I’ve known where leading statisticians have disagreed about how a particular set of results should be interpreted, or when the stats have been conveniently re-interpreted to produce a different, more desirable, conclusion.
There is also the danger of inappropriate causation.
A particular factor might appear to be producing a certain effect, but that effect may actually be the result of another influence. For example, the simple action of doing something (i.e. anything) differently has been shown to produce a significant increase in work-rate and successful task-completion.
Quantitive versus Qualitative:
While quantitive scientific research deals with the objective and reliably repeatable under identical conditions, qualitative educational research deals more with subjective observations and opinions. However, to justify its conclusions, it often adopts techniques and methodologies that are based on sampling, surveys and scale-based ratings that produce numerical data that can be subjected to statistical analysis – which then runs in to the same problems of manipulation as scientific research.
Some educational research involves the extended use of data-free case-studies and ethnographic studies: while there is a potential for considerable bias in these, they can result in interesting, thought-provoking insights for further consideration.
Neuroscience and Cognition:
Meanwhile a current controversial area of education research is based in the area of neuroscience and cognitive psychology which explores the biological process of the learning. Unfortunately the ‘scientific’ data derived from such studies has sometimes been used to sell various educational products into schools, while other findings have been misunderstood.
As a result there are those who support such research and see at as a useful advance in understanding how children learn, while others dismiss the findings as myths that should be ignored and discredited.
Yet despite all these pitfalls, reformers and governments impose policy based on the data, which also feeds the media. There are no magic silver bullets in education that will instantly create better teachers and better students.
And that’s most important to be aware of at a time when the DfE are busy reforming education based on research data that they choose to interpret in particular ways that suit their policies. Indeed, while in the past policy arose out of the results of research, today it’s often the other way round – research is undertaken to justify a particular policy.
Image: Shutterstock
The Problems:
Perhaps all these problems involved in conducting meaningful, but unreliable educational research, explains ‘why’ that in well over a hundred years of the provision of education, there is still little consensus as to what and how we should be teaching our children. While research in most other aspects of our lives has succeeded in producing a sense of progress, the current political policy is that we need to go back to traditional approaches to education that have little to do with the future learning needs of our children.
Why has the educational research that has been undertaken failed to provide us with any widely accepted sense that we have discovered new and improved ways of educating our children? Is there anything to suggest that future research will do any better?
Examine and Discuss:
If all this makes it sound as if most educational research is a waste of time, what it does very effectively achieve is to cause us to examine and discuss existing practice, raise questions and search for possible answers. The evidence it produces points the way to new or emerging practices and methodologies that might in certain situations prove to be worthwhile for large or smaller groups of the student population.
The most beneficial aspect of much of the research I have been involved with has been to those who have actually been involved in collecting and analysing the data – indeed it’s probably the best form of CPD there is. Just as long as it’s understood that it isn’t going to provide any guarantee or certainty of improvement for everyone.
So, if someone pushes a piece of educational research in front of you, read it with interest and with an open-mind. Assume that ‘the research’ is not necessarily a universal truth, and that the claims it makes might not be completely trustworthy or relevant to your teaching or school.
What is important is to be able to evaluate the evidence behind the claims of educational research, so below is a handy cut-out and keep checklist. Although it relates to Scientific rather than Educational research, you may find it helpful if you are doing your own research to make sure it will pass the test when others come to challenge it!
Credit: CompundChem
Download ‘How to spot unreliable educational research’ here.
End.
Tristram Shepard writes for Teacher Toolkit. You can read more of Tristram’s articles here and blogs via All Change Please.
He can be found on Twitter at @TristramShepard.
For example… Use, misuse and abuse of research in the education White Paper by Ian Menter https://www.timeshighereducation.com/blog/use-misuse-and-abuse-research-education-white-paper
There is no correlation between piracy and global warming – this represents the coincidence of two single data points (1. increase in piracy, 2. increase in warming), not the dependency of two large sets of random data points (e.g. piracy up in 1745, warming down in 1746 etc). While it is true that a correlation between piracy and global warming (if one could be shown) would not imply that one caused another, it would imply that either one caused the other or a third variable caused both. Not only is the popular aphorism “correlation does not imply causation” not true, it is deeply misleading because correlation is the only secure evidence for causation – read David Hume on the subject.
In short, the problems with research highlighted by this article nearly all refer to poor practice by educational researchers and teachers, and not with the principle of (quantitative) research itself.
The key problems that are relevant are the number of variables involved, the clustering of students (not mentioned), and the difficulty of implementing research conclusions consistently. Combined with the fourth problem – the widespread ignorance among both teachers and (bizarrely enough) education researchers themselves about the basic principles of qualitative research, the answer to all these problems is education technology. See my http://edtechnow.net/2015/09/10/red15/.
Thanks, Crispin.
And thank you bocks1 for your kind & thoughtful response.
I completely agree with you that the whole scientific/technological approach to education (i.e. asking “what works?) counts for nothing unless we can be clear about our objectives (which we are not on the whole). I have written another piece about this at “The purpose of education” at http://edtechnow.net/2016/02/20/purpose/. One point, which I cover in this article, is that it is not primarily the role of teachers to decide what the purpose of education is – as it is not the role of any service provider but rather the customer to decide on the purpose and objective of the service. We may question who precisely is the customer in the case of education – but we can be sure of one thing, that it is not the service provider.
On the difficulty presented by number of different variables involved in education and the related issue of science vs art, I think that in almost any branch of technology (e.g. medicine or civil engineering) there is plenty of scope for individual initiative. Humans are not so very different from each other that there are not some pretty good generalisations (or perhaps we could call them principles) that can be established by aggregating observational data – but the question of applying those principles back to a specific case requires considerable experience, perception and interpretation. Technology is rarely about following a script. The principles might be written in a book but the situation to which the principles have to be applied – what is in front of the eyes of the teacher – never is. So I believe that the talk of automating teachers out of existence is nonsense, which is a reason I have argued that MOOCs are not going to work (http://edtechnow.net/2012/12/29/moocs-and-other-ed-tech-bubbles/).
As for “distilling that stuff” – I think that is precisely what technology is, canned expertise/research. Which (as I say above) will not replace but rather complement uncanned expertise (teachers). I would think of edtech as providing the “tools of the trade” on which any professional relies. The question then is – who will produce the education-specific software systems that will do this?
Best, Crispin.