Begin dit jaar schokte het boek  Academically Adrift: Limited Learning on College Campuses de Amerikaanse publieke opinie. De centrale stelling in het boek is dat een groot deel van de nieuwe studenten van de Amerikaanse universiteiten niet zo verschrikkelijk veel leert gedurende hun studietijd. De auteurs presenteerden een onderzoek onder 2300 studenten van 24 universiteiten over vier jaar tijd. Het onderzoek richtte zich op de verbeteringen die de studenten bereikten in kritisch nadenken en het verslag doen daarvan in schriftelijke papers, gerelateerd aan de hoeveelheid studie die ze verrichten en de schrijfvaardigheden die ze hadden en verder ontwikkelden.

Richard Arum, co-auteur van het boek, professor in Sociologie aan de New York University stelt dat ‘the fact that more than a third of students showed no improvement in critical thinking skills after four years at a university was cause for concern. … Our country today is part of a global economic system, where we no longer have the luxury to put large numbers of kids through college and university and not demand of them that they are developing these higher order skills that are necessary not just for them, but for our society as a whole’. Een andere uitspraak is er een die ook voor het Nederlandse Hoger Onderwijs wel eens is gebruikt: ‘There’s a huge incentive set up in the system [for] asking students very little, grading them easily, entertaining them, and your course evaluations will be high’.

Een reden voor het achteruitgaan van de vaardigheid in het kritisch nadenken is volgens de auteurs de afname in de academische ijver en studielast: 35 procent van de studenten stelde niet meer dan 5 uur per week te studeren en 50 % zei dat dat ze in het vorige semester geen enkele module hadden gehad waarin ze meer dan 20 pagina’s aan papers hadden in te leveren.

‘At every university, however, there are students who defy the trend of a decline in hours spent studying — and who do improve their writing and thinking skills. Our study found this to occur more frequently at more selective colleges and universities, where students learn slightly more and have slightly higher academic standards. Overall, though, we found that there has been a 50 percent decline in the number of hours a student spends studying and preparing for classes from several decades ago’, zo stelt Arum. ‘If you go out and talk to college freshmen today, they tell you something very interesting. Many of them will say the following: ‘I thought college and university was going to be harder than high school, and my gosh, it turned out it’s easier’.

Maar klopt het onderzoek wel ?

Daar wordt over getwijfeld. Alexander Aston, professor emeritus Higher Education and Organizational Change aan de University of California in Los Angeles, maakt korte metten met de studie. ‘I should note the authors’ baffling failure to report certain other basic data that any reasonable reader would like to know. For example, nowhere do they indicate how many students (or what percentage) showed any degree of improvement. Nor do they report each student’s actual scores on the test, but only whether significant improvement was made. Given that people familiar with the Collegiate Learning Assessment might like to know how much improvement individual students showed, it is difficult to understand why the authors provide no information on the distribution of actual scores, which would indicate what proportion of students showed different amounts of improvement from their generalization. Indeed, they also fail to report how many students’ scores declined (or by how much), something that would certainly be of interest to educators. With such a large sample, certainly there must have been at least some students, for example, whose scores got worse’.

En: ‘The method used to determine whether a student’s sophomore score was ‘significantly’ better than his or her freshman score is ill suited to the researchers’ conclusion. The authors compared the difference between the two scores—how much improvement a student showed—with something called the ‘standard error of the difference’ between his or her two scores. If the improvement was at least roughly twice as large as the standard error (specifically, at least 1.96 times larger, which corresponds to the ‘. 05 level of confidence’), they concluded that the student ‘improved’. By that standard, 55 percent of the students showed ‘significant’ improvement—which led, erroneously, to the assertion that 45 percent of the students showed no improvement.  …  The first thing to realize is that, for the purposes of determining how many students failed to learn, the yardstick of ‘significance’ used here—the .05 level of confidence—is utterly arbitrary. Such tests are supposed to control for what statisticians call ‘Type I errors’, the type you commit when you conclude that there is a real difference, when in fact there is not. But they cannot be used to prove that a student’s score did not improve’.

Astin gaat door met allerlei andere argumenten om het onderzoek onderuit te halen en hij sluit af met de volgende algemene conclusie: ‘In short, these considerations suggest that the claim that 45 percent of America’s college undergraduates fail to improve their reasoning and writing skills during their first two years of college cannot be taken seriously. With a different kind of analysis, it may indeed be appropriate to conclude that many undergraduates are not benefiting as much as they should from their college experience. But the 45-percent claim is simply not justified by the data and analyses set forth in this particular report’.

Met andere woorden: het zou best eens kunnen dat de onderzoekers gelijk hebben, maar dan moeten ze wel hele andere onderzoeksprocedures en andere analysetechnieken gebruiken, want de gebruikte onderzoekstechnieken kunnen niet door de beugel….