Given that most of the work on Learning Design exists outside the US, Ken asks whether there are any particular barriers to adoption of Learning Design software that I've encountered in the US. I don't have a clear answer to this, but I'll pick on one of the factors that most worries me about US education.
Automated Testing.
I find the extensive use of automated testing in the US amazing, especially in K-12. Some of the most important lessons of education cannot easily be tested in an automated way - for example:
- the ability to hear arguments other than those you already believe and consider these carefully (and potentially change you view);
- the ability to work in teams to think creatively about solving a problem;
- the ability to express your ideas clearly in written or oral form
- the ability to research a new problem to find out what is already known about it so that you can approach the problem with greater knowledge than you can achieve by thinking on your own;
- an understanding of an individual's role in society, and the interconnection of business, the environment, politics and culture;
- an appreciation of beauty, music and art;
- a sense of the lessons of history for modern dilemmas;
- an understanding of the development of science and its strengths (and limits);
- an ability to understand and contrast cultures and religions other than your own;
- an understanding of your own ethics and values, and how these relate to those of others;
. . .and the list could go on. The point is that many educators would agree that a rich education should achieve learning of the kind described above, not just memory of the facts that can be tested in a multiple choice quiz. And it is important to note that it is possible for a teacher to assess learning of the kind outlined above, but not via a quiz (and also not perfectly - but see comments below on reliability).
The assessment required for the learning described above is often formative, not just summative; a dialogue between student and teacher, not just a judgement; and most importantly, time consuming for a real human being (the teacher), not a process that can be outsourced to a machine. In essence, it is an attempt at authentic assessment.
Some of the pedagogical approaches that are best supported by Learning Design (as compared to other e-learning approaches) may not fit with a culture of automated testing. And given that students will focus their learning on the methods used to assess them (and increasingly teachers simply “teach to the test”), then I sense there are structural barriers to a greater realisation of the benefits of a Learning Design approach that arise from US assessment practices. The frightening dimension of this is that if our students only learnwhat we can test via automated testing, then they may not become the well-rounded people we hope to see graduate from our educational systems. This may ultimately be detrimental to our society and our world.
I see two arguments in favour of retaining extensive automated testing - one that I consider to be invalid, and one that is somewhat valid.
The invalid argument is the classic “reliability and validity” arguments from educational measurement and test theory. The argument is that automated tests are a fair judge of a student's ability, whereas the kind of assessment needed for the types of learning described above will be subjective and unreliable. For now I won't dispute the second part of this argument, but in terms of fairness arising from reliability of automated assessment, there is a fundamental problem with this argument that is rarely discussed.
Educational measurement, if it is to be valid, needs to meet the requirements of “scientific” or true measurement. Scientific measurement requires that the underlying attribute being measured (in this case a student's ability in a particular area) is quantitative (like length) and not qualitative (like colour). For an attribute to be quantitative, it is not simply a matter of assigning numerals to things, rather, a scientific study to investigate whether or not the underlying attribute has the “structure” required for something to be quantitative needs to be conducted.
For something like length, this is easy to establish, as we can compare and add lengths. For other attributes (such as density, or potentially educational abilities), we can't add objects/people together, but we can potentially order them. The discovery of conjoint measurement provides a method of testing ordered structures to see if they are also quantitative.
So if one applies the rigorous requirements of scientific measurement to educational scores, what do we find? Well, when I last looked into this field deeply*, there was no robust evidence that educational measurement is quantitative. If this is the case, then we can't add scores together in education and achieve at a meaningful outcome (eg, creating an “overall” score is invalid, because the numerals being added together aren't based on a demonstrably quantitative attribute). And if this is the case, then we don't actually have fairness, as the reliability and validity that we appear to have are built on a false foundation.
*For a detailed version of this argument, see Dalziel (1998) 1
If automated testing produces scores which are not real measurement, but rather spurious numerals; and given that the use of automated testing has such a great impact on the way students learn (and how teachers teach), then I believe there is an argument for a fundamental change in the way education is conducted in the US (and elsewhere). If automated testing is rejected, and the types of learning described above are valued, then the alternative approach to education could look more like typical Learning Design sequences.
The second, somewhat valid defence of extensive automated testing is that any alternative to this would involve enormous human effort on the part of educators. If educators need to conduct rich assessments with feedback and dialogue for each individual student, then this would take an enormous amount of time; and educators are already incredibly busy, so it is hard to see where this time could come from.
I agree that it would take a lot of time, and that teachers are already very busy, but ultimately I think the current alternative is worse. If student are mostly just memorising for automated tests, and then forgetting almost everything they memorised soon after the test, then the educational process is not achieving much real learning anyway. Given this, I think we could change our educational processes to focus on less content delivery (and hence less fact testing), and spend more time on the types of learning outlined above.
I hasten to add that I'm not advocating content-free education - far from it - it is only through a rich engagement with real content, real events, real discoveries, that the broader types of learning will come alive and be retained by students. But by changing assessment practices, and giving much more time to this element of education, we change the way that students learn (and the way teachers teach), and may have a better chance at achieving these broader types of learning.
While Learning Design could help with more authentic learning and assessment tasks, it could also help with educators' lack of time. Instead of the inefficiencies of each educator around the world re-inventing the wheel for commonly taught topics, the re-use of existing “good practice” Learning Designs could reduce preparation times, and hence free educators to spend more time on authentic and individualised assessment.
I believe this is a dream worth fighting for, and I sense I'm not alone.
- 760 reads
