MERGA

How Knowledgeable is ChatGPT 4o? Assessing the Pedagogical Content Knowledge of a Generative Artificial Intelligence Tool

  • Home
  • >
  • Publications
  • >
  • How Knowledgeable is ChatGPT 4o? Assessing the Pedagogical Content Knowledge of a Generative Artificial Intelligence Tool
Author: Scott Cameron, Carmel Mesiti & Paul Fijn
Publication Year: 2025

Abstract

Generative Artificial Intelligence (GenAI) chatbots provide teachers with opportunities to enhance their expertise by exploring topics relevant to their practice. However, studies have raised concerns about ChatGPT’s mathematical accuracy. This pilot study evaluated ChatGPT’s Pedagogical Content Knowledge (PCK) and Content Knowledge (CK), revealing that while it generates varied and detailed responses indicating competence in PCK and CK, some responses are flawed or incorrect. While ChatGPT 4o can serve as a valuable professional learning resource in certain areas of mathematics education, traditional teaching methods and human insight remain essential.

Type: Research Paper
  Download Publication Downloaded 2 times

Short Communications & Round Tables

The following documents are essential to read and complete when considering submitting a short communication or round table to the MERGA conference (available in the Submission section of this website).

  • MERGA Paper Template – to be used to write the abstracts which are then submitted to the MERGA conference website
  • MERGA Publication Agreement – to be submitted on the MERGA conference website at the time of the abstract submission

Submissions not exceeding one page are required for short communications and round tables. The submissions must be prepared using the conference template, and can include essential references. They will be reviewed by the Editorial Team and, if accepted, will be published in the conference proceedings as one-page abstracts (not as papers). Presenters are invited to prepare a paper for distribution at the conference, but these papers will not be included in the proceedings.

Short communications are suitable for reports on research in mathematics education that do not fully meet the requirements for published papers. These might include works in preliminary stages, reports of pilot projects, initial reviews of literature, ideas or suggestions for future study, and briefer discussions of particular issues. Short communications allow new researchers to obtain feedback on projects in a constructive and supportive environment, and foster the building of links between researchers with similar interests.

Short communications are presented by author(s) only, allocated half of the time for research reports (in past, this was 20 minutes). At least 5 minutes is to be allocated for audience questions and open discussion.

Round tables are suitable for presenters seeking involved interaction with the audience in relation to their research or topic of interest in mathematics education, and for those sharing their insights and advice with the early career colleagues. These might include discussion of an emerging topic, co-analysis of provided student work, solving a mathematical task, or discussing the demands and benefits of reviewing for high quality journals. Round tables allow finding peers with similar research interests, exploration of new research avenues, and building the capacity of MERGA community.

Round tables are led by author(s) only, allocated the same time as research reports (in past, this was 40 minutes). The abstracts should make clear the interactive element of the proposed activity, with no more than 15 min of the session time in a presentation mode.

Beth Southwell Practical Implications Award

The Beth Southwell Practical Implications Award (BSPIA) recognises high-quality mathematics education that produces insights for the teaching profession and/or student learning.

The award consists of $500 and a plaque to be presented at the Conference.

Nomination process 

There are two ways a paper can be nominated for the BSPIA:

  • Self-nomination: When you submit your conference paper, check the box that asks if you would like to apply for the BSPIA.
  • Nomination via peer-review: Anyone who submits a Conference paper for peer-review will be considered for nomination by the reviewers. 

Single and co-authored papers are eligible for consideration.

When you write your paper, please ensure that you observe all general paper submission requirements including the maximum page length.

Judging process

Submissions must be deemed eligible for publication in the Conference proceedings by the initial reviewing panel. Submissions accepted for presentation only will be excluded from consideration.

The judging panel will consist of two MERGA members and two AAMT nominees and will be chaired by the VP Development.

The judging criteria are:

  • Identification of a persistent and significant research problem
  • Synthesis of recent research literature and relevant policy initiatives
  • Robust methodology producing valid, reliable findings
  • Insightful discussion of practical implications for the teaching profession and/or student learning
  • Clear, succinct style of academic writing

Winners are notified four weeks prior to the Conference and are invited to present a keynote at the annual conference.

The Beth Southwell Practical Implications Award page on this website provides further information, including the history of the award and recent winners.  

Research Papers

The following documents are essential to read and complete when considering submitting a research paper to the MERGA conference (available in the Submission section of this website).

  • MERGA Paper Template – to be used to write the papers which are then submitted to the MERGA conference website
  • MERGA Publication Agreement – to be submitted on the MERGA conference website at the time of the paper submission

Research papers can take two major forms: 

1. Reports of empirical investigations 

When empirical investigations are reported (such as in an experimental intervention, confirmatory study, or action research, etc.), the paper should also include

  • a statement of rationale for methodologies used in collecting and analysing data;
  • a critical discussion of data findings in the light of the research literature; and
  • in the literature review, prior work in the area should be acknowledged and an explanation of how the work reported in the paper builds on that earlier work should be included.

2. Reports that are not based on empirical research including:

  • a theoretical discussion;
  • a position paper;
  • a report of scholarly enquiry in progress;
  • a literature review, a meta-study;
  • an account of a new initiative;
  • a reflective critique of practice; or
  • any mixture of these or other recognised scholarly forms.

When the work is a theoretical discussion, a position paper, a report of scholarly inquiry in progress, a review of literature, a theoretical study, a meta-study, an account of a new initiative, a reflective critique of practice or any mixture of these or other recognised scholarly forms, the material presented must be discussed critically, and alternative points of view relating to themes presented should be appropriately argued.
It is expected that presenting authors will have 40 minutes to present their work at the conference. At least 10 minutes must be allowed to field questions and comments from the audience.

Structure of research papers – All papers for publication in the conference proceedings should contain the following:

  • a statement of the problem/issue and a discussion of its significance;
  • a critical analysis of the research literature as it relates to the topic of the paper; and
  • conclusions and implications for mathematics education derived from the study.

All papers must respect MERGA’s ethical guidelines relating to research work. Papers should not be more than the set length. In addition, papers must be: readable; free of grammatical, spelling and typographical errors; and adhere strictly to style requirements advertised by the conference proceedings Editorial Team.

Originality – Only research papers that are substantially different from work that has been published previously will be considered for publication in the conference proceedings and/or presentation at the conference.

Reviewing of research papers – Research papers will be blind reviewed by a panel of peers approved by the conference committee. The main purpose of the refereeing process is to contribute to the growth and development of quality practice in mathematics education research. Thus reviewers are asked to assist authors by providing helpful feedback and to comment on the suitability of papers for presentation at the conference. Accordingly, it will not be assumed that published papers presented at the conference will be as polished as articles in scholarly journals. Referees will be asked to assess papers being reviewed against the accepted norms for scholarly works presented at MERGA conferences, as set out above.

Each conference proceedings’ Editorial Team will exercise discretion over the reviewing process. Reviewers’ comments will be returned to the authors. Authors whose papers are not accepted for publication may be invited by the editorial panel to present their paper at the conference as a Short Communication, with a 1-page abstract (which they will be invited to provide) being published in the proceedings. Papers may be rejected outright, with no opportunity for presentation at the conference in an alternative form.

Research Symposia

The following documents are essential to read and complete when considering submitting a research symposium to the MERGA conference (available in the Submission section of this website).

  • MERGA Paper Template (to be used to write the papers which are then submitted to the MERGA conference website)
  • MERGA Publication Agreement (to be submitted on the MERGA conference website at the time of the paper submission)

Presentation of groups of published papers related by theme in the form of a research symposium is encouraged. The symposium forum will be particularly suitable for presentations relating to a single large project or presentations that explore topical themes from different and/or related perspectives. Special Interest Groups [SIGs] are encouraged to consider the symposium option as a means for sharing and discussing current research.

A symposium should consist of no more than four presentations of about 15 minutes duration each. The written papers should be half the length of research papers as described for research papers. Both research report types – empirical or non-empirical – are acceptable as published symposium papers.

A brief overview of the symposium (limit one page), including a symposium title, an introduction to the theme/project, and a short introduction to each of the 3-4 contributions, must be submitted with the set of short papers. Please list the symposium convenors as the authors on the first page, and name the paper authors in the text description.

The symposium proposer will also nominate a person to chair the symposium, and a discussant can also be named if desired. This information should accompany the collection of papers submitted for review.

The set of symposium papers (and the overview) will be blind reviewed by a review panel. The main purpose of the reviews is the same as for published papers, and the same criteria are used. The reviewers will be asked to consider the cohesiveness of the set of symposium papers. They will indicate whether the symposium as a whole, and each paper within it, should be “accepted”, “rejected” or if it “requires revision”. If it is deemed that one, some or all of the papers are in need of revision, the reviewers will outline which papers need to be revised and provide suggestions for the required changes. When the revisions are made, the symposium papers will be re-submitted and the set of papers will be sent to the same review panel for further consideration. As with research papers, the final decision about which symposium papers will be published is at the discretion of the Editorial Team.

The date for submission of the collection of symposium papers is the same as for Early Bird papers. This date has been set for the benefit of the group of authors of symposia papers. Should the symposium papers require revision, the authors will have the time to make the corrections and resubmit the set of symposium papers to be re-reviewed by the original reviewers.

Presentation of symposia: Symposia are presented by author(s) only, usually within a 90 min block. At least 10 minutes must be allocated for audience questions and open discussion.

Early Career Research Award

In order to encourage new researchers in mathematics education, MERGA sponsors an award to an author in the early part of her/his career. The award, for excellence in writing and presenting a piece of mathematics education research, consists of a plaque and a prize of $500 and is presented at the annual conference. 

Applying for the award

Entry for the Early Career Research Award is by submission of a written paper for presentation at the conference through the Early Bird process. Conditions of eligibility, information about the judging process, and the criteria judges will observe are indicated below. If you are applying for the Early Career Research Award, please ensure that when you upload your paper on the conference website, you also send an email to the Conference Secretariat indicating that you are an entrant for the Early Career Research Award. Note that at some MERGA conferences there is also a form to complete or a box to tick on the registration form, so check the conference website carefully. Please note that co-authored papers ARE NOT eligible for entry into the Early Career Research Award, nor are Round Table or Symposium papers.

Rules and eligibility for the Early Career Award

The Early Career Research Award page on this website provides further information about this award, including a list of recent winners.

Early Bird Review Process

The Early Bird review process is a form of mentoring, principally for new researchers. However, anyone is eligible to make use of it. Research papers submitted through the Early Bird process must be received by the Early Bird due date (i.e., the closing date in January). They must meet the requirements as set out for MERGA Research Papers. Early Bird papers should be uploaded following a link on the conference website. Authors will be asked to create a login into Oxford Abstracts (our conference management system) and submit the blinded file (anonymised) in the correct template for review.

Early Bird papers undergo a double-blind MERGA reviewing process. There are three possible outcomes of the review, and actions the authors need to follow.

  1. When the paper is accepted (for presentation at the conference and publication in the proceedings), the authors will be asked to provide their full unblinded manuscript and publication agreement.
  2. When small revisions are required, the revised papers will need to be resubmitted by the main submission deadline in March. The changes are considered by the editors, and the papers are not usually sent out for review again. The editors decide whether the paper is accepted for publication as well as presentation at the conference.
  3. When more major revisions are required, the reviewers will provide the author/s with feedback on how to how to strengthen the paper. The paper will need to be resubmitted by the main submission deadline in March, and it will be sent out for a new double-blind review.

Authors are notified of the outcome as soon as possible (usually within a few weeks, and in time for resubmission). Letters are sent to authors to indicate (a) accepted for publication, (b) small revisions required, (c) or major rewriting required.