The singer-songwriter Paul Kelly (1992) has a song titled Taught By Experts in which the narrator laments the cynical nature of learned behaviour. He rarely sings the song’s last stanza, in which he changes one word of each line of the chorus of Bob Dylan’s Like a Rolling Stone. Dylan famously borrowed phrases from folk singers he admired, Kelly was sarcastically demonstrating how you could really get it wrong if you were not an expert yourself. Advances in social media have changed the narrative of what an expert is, whereby impact on the profession is best gained with a successful YouTube channel or Blog. At the same time, “evidence-based” has become a phrase commonly (and loosely) used by anyone with an opinion about education. It is increasingly challenging to counter this new wave of “expert” opinions, especially when they gain political and general-public momentum.
It is unsurprising that views about what constitutes rigorous and “best-practice” research differs across and within disciplines, even by conventional academic experts (Lowrie, 2024). The Australian Education Research Organisation (AERO, 2022) maintains that rigorous evidence occurs when research methods isolate the actual impact of an approach. Most folk in our education community would agree with that. However, AERO go on to say that when evidence does not fit within their defined confidence levels, they will draw on “expert research guidance to make an assessment about how confident we are in the effectiveness of an approach” (p. 2). Presumably, AERO decide what research is worthwhile based on their confidence in the expert. Typically, research organisations such as the National Health and Medical Research Council (NHMRC; 2000) are more explicit about the approaches that demonstrate the highest levels of evidence. For them, and many other organisations, “gold standard” research includes meta-analysis, systematic reviews of literature, and randomised control trials (RCTs). Noteworthy, the weakest form of evidence is expert opinions.
For our discipline, and profession more generally, it is becoming increasingly challenging to counter the wave of experts demanding changes to the curricula and pedagogical approaches based on expert opinions (at best). However, we do not seem to possess the research rigour to counter the arguments, given our field rarely conducts this work. In a review of publications over the last ten years published in two of the leading journals in our field, less than 5% include research designs that would meet the “gold standard”. This presentation considers some of the challenges faced by the mathematics education community in a changed environment where the term evidence is used carelessly, and evidence-based practices are ill-defined. “Expert” opinions dominate the discourse, rather than studies with rigorous methodological design. This presentation examines how well prepared our discipline is to counter such challenges