High-quality K-12 schools and healthcare systems are probably the two most important components of the welfare state. Which leads to the obvious question: what is “high-quality”?

In the context of schools, the OECD’s Program for International Student Assessment (PISA) has provided well-accepted comparative measures for school systems at the upper-secondary level. PISA also provides provincial level results. Two assumptions underlie PISA surveys. First is that student mastery of upper-secondary-level reading, mathematics, and science is, world-wide, required for meaningful participation in modern society. Second is that low-stake standardized surveys of student learning outcomes are a necessary, (but insufficient) measure of school system performance.

An unavoidable difficulty in assessing student outcomes is that school quality is not the only relevant factor. The most important factor is often the student’s family.¹ The contribution of family members toward student learning depends on many things: parental education; parental motivation for children to pursue academic studies, which may be influenced by ethnic and religious expectations; and parental income, which affects parental ability to devote time and financial resources to children’s studies (Gang, Zimmermann 2000; Ludeke et al. 2018; Stokes 2008). Peer effects also matter (Richards et al. 2008). In a school where the majority are from families with high education expectations, students perform better than in schools where the majority are from families with low education expectations.

The criteria for defining good schools include more than student performance in core subjects. Schools are institutions for transmitting society’s culture: history, literature, and traditions, and for imparting shared cultural norms necessary for civil society to function. These additional dimensions are complements to—not substitutes for—reading, mathematics, and science.

Overall, in all three subjects, Canadian schools have performed well above the relevant OECD averages in each round of the PISA surveys, starting in 2000. In the 2018 round, out of 78 participating jurisdictions, Canada ranked 6th in reading, 12th in mathematics, and 8th in science. Since Canada has consistently maintained an enviable overall ranking, there is a danger of complacency. Our system is good, but has weaknesses.

Since benchmarking of average international student performance in the 2000s, average OECD scores among the original OECD countries have been relatively stable over successive rounds.² However, Canadian scores in all three subjects have declined from the respective benchmarking year to 2018: reading has declined from 534 (benchmark year 2000) to 520, mathematics from 532 (benchmark year 2003) to 512, science from 534 (benchmark year 2006) to 518.

The first section reviews provincial outcomes. In mathematics, Quebec is a positive outlier. I offer tentative explanations for its exceptionalism. The second section discusses equity of outcomes, in terms of socio-economic family conditions and Indigenous/non-Indigenous outcome gaps.

Table 1: Canadian PISA 2018 Performance, Score and Changes, by Province

For the three subjects, Table 1 provides a snapshot of provincial performance. It shows 2018 average provincial scores and two calculations of difference: from the benchmark year for each subject to 2018, and the recent change from the 2015 to 2018 round. The provinces are ranked by 2018 reading scores. What are the key conclusions to draw?

It is useful to categorize the provinces into three groups:

• Group A – Ontario, Quebec, British Columbia, Alberta: The four large provinces (with population over four million) are home to 86 percent of Canada’s population. For all subjects, they have the four highest 2018 scores.
• Group B – Nova Scotia, Newfoundland, Prince Edward Island: While outcomes are weaker than provinces in Group A, average results in this group are higher than provinces in Group C, and have been more stable than in Group C.
• Group C – Saskatchewan, Manitoba, New Brunswick: With one exception, the highest subject score among these three provinces is below the minimum comparable score in Group B. Furthermore, all but one of the benchmark-to-2018 declines in this group are statistically significant.³

On all three subjects, the Group A provinces score sufficiently high that they are statistically above the OECD average in all subjects.⁴ Among the six smaller provinces, this is not the case. To summarize: among the 18 Group B and C scores, nine scores are statistically above the OECD average; eight are, in terms of statistical significance, at the OECD average; one score is significantly below average.

Perhaps the explanation for weak scores among Group B and C provinces is that these provinces spend less on education than the others, which in turn could be a partial explanation. This is not a persuasive argument. Average per student spending (unweighted by population) among the four Group A provinces in 2014/15 was $12,600, lowest among the three groups. Among Group B provinces, average spending was $13,100, among Group C provinces $14,600 (Statistics Canada 2018).

A more persuasive explanation is that scale economies permit Group A provinces to achieve superior student outcomes despite spending less per student. At the secondary level, the cost per student of delivering comparable education quality in large cities is much lower than in small towns and rural areas. All of Canada’s cities with a population over one million are in a Group A province. Provincial school systems acknowledge scale economies via more generous per student funding formulas in rural schools. Nonetheless, the scope of course offerings in rural schools is usually less than in urban centres.

A third potential explanation is differing socio-economic conditions across provinces. PISA generates an index of the socio-economic status of a student’s family, using evidence provided by the participating student.⁵ Regression analysis (admittedly very crude given 10 provincial observations) suggests that both scale economies and socio-economic status matter.

Mathematics is the subject in which Canada’s decline from the benchmark year to 2018 is most pronounced. In all provinces except Quebec and Prince Edward Island, the decline has been statistically significant.⁶

Writing for the C.D. Howe Institute, Anna Stokke (2015) makes two arguments. She concludes many provincial curricula have adopted “discovery based” mathematics instruction, a teaching strategy that invites students to discover independently solutions to problems and discourages direct teaching by instructors. It is an inefficient strategy, she argues, that generates weak results at the early primary level, a disadvantage that many students carry into the secondary level. It encourages teachers to ignore the role of early grade memorization (such as multiplication tables) and of simple algorithms (such as the carry-over rule for subtraction), and minimizes pencil and paper practice. Recent revisions to the Ontario elementary mathematics curriculum are an attempt to address the problems Stokke raises (Ontario 2020). Second, she emphasizes the importance of instructors’ subject knowledge. It is here that Quebec outperforms the other provinces.

Paul Bennett’s (2018) explanation of superior Quebec performance is that secondary-level mathematics teachers are required to take more rigorous university mathematics training than is the case in other provinces. Also, Quebec students enter secondary school in Grade 7, as opposed to Grade 9 in many of the other provinces. The combination of better-trained secondary math teachers and more years of secondary-level instruction is, Bennett concludes, the basic explanation for Quebec’s superior performance. The trade-off, he acknowledges, is somewhat lower secondary completion rates:

A third explanation is the prominent role of private schools in Quebec. In Quebec private schools, the average PISA 2018 mathematics score was 579, in public schools 519. Much of this gap is attributable to the above-average socio-economic status of families that select private schools. Despite loss of positive peer effect in public schools due to the choice of a private school by many high-status families, the public school score is above the Canadian average of 512.

The second set of differences, between the PISA rounds in 2015 and 2018, are for most provinces small and not statistically significant. The major observation here is that British Columbia is no more one of the best-performing provinces. From the 2015 round, it experienced statistically significant declines in all three subjects.

A partial explanation in the case of BC may be the underlying strategy of curriculum design that seeks to minimize any memorization: “The redesigned curricula are described as concept-based and competency-driven. They place more emphasis on the deeper understanding of concepts and the application of processes than on the memorization of isolated facts and information” (BC 2020). It may be time for BC to do as in Ontario and reintroduce traditional teaching strategies in mastery of basic arithmetic.

Socio-economic equity

An implicit goal of Canadian education policy is not only achieving high average scores, but also minimizing expected decline in outcomes among students as socio-economic conditions decline from top to bottom scores.

There exist many measures of equity in education outcomes. The measure favoured by PISA is: After ranking students by the PISA socio-economic-cultural index (ESCS), equity is measured by the difference between the average scores among the top-quarter and among the bottom-quarter. The smaller the difference the more egalitarian the system.

At the provincial level, with a few exceptions, the Group A provinces rank highest among both the top- and bottom-quarter results. The three lowest top- and lowest bottom-quarter averages are all in Group C provinces.

Indigenous education outcomes

The gap in overall PISA Canadian reading scores between top- and bottom-quarters is 68, more than 0.5 standard deviation. The gap between non-Indigenous and Indigenous PISA scores is probably of similar magnitude. (O’Grady et al. 2019, 27).

In search of better information on Indigenous education outcomes, education ministries of six provinces—the four western provinces plus Prince Edward Island and Newfoundland/Labrador—agreed to add a voluntary Indigenous identifier question to the Canadian 2018 PISA questionnaire. The agency responsible for Canada’s contribution to PISA is the Council of Ministers of Education, Canada (CMEC). Unlike Australia and New Zealand, two other Commonwealth countries with sizeable Indigenous populations, CMEC decided not to publish the Indigenous results and, instead, to make data available to the provincial ministries only for internal use. The data are supposedly not representative of the Indigenous populations within each participating province.⁷

The decision not to release Indigenous results invites speculation that rationales other than sample bias lie behind the decision. There is a strong tradition in France not to identify any official statistics in terms of ethnicity, on grounds that those with racial bias will exploit weak outcomes to disparage ethnic minorities. Probably, this ideological opposition figured in Quebec’s refusal to pose an Indigenous identifier question in the Quebec sample – and it may have been important among the three other provinces that refused to add the identifier.

Against the argument of misuse of data is the old maxim, “if you don’t know where you are, you are unlikely to get where you want to go.” The decision to withhold results flouts one of the fundamental rationales for conduct of PISA surveys, namely the desire to provide a better empirical foundation for public discussion of education policy.

The Canadian K-12 system ranks high in international comparison – but has weaknesses. The first is the growing gap between results in the four large provinces and the six smaller provinces.

Since scale economies are a factor in explaining gaps in provincial performance, there is a strong case for Group B and C provinces partnering with other provinces in order to realize scale economies. They could share curricula reforms and encourage their university education faculties to develop complementary specialties. Sharing curricula reforms does not mean copying the curricula of other provinces displaying weak results.

Indigenous/non-Indigenous education outcome gaps are large – albeit the evidence is fragmentary. Provinces should be undertaking ambitious programs to improve Indigenous outcomes, in provincial schools. Two-thirds of those identifying in the Census as First Nation live off-reserve, as do all Métis. Combined, four-fifths of Indigenous families live off-reserve, and their children attend provincial schools (Richards 2018). Given these location realities, the provinces, particularly Saskatchewan and Manitoba, should be taking the lead in closing the Indigenous/non-Indigenous gaps in core subjects. First Nations should be undertaking similar outcomes on reserve schools. This undertaking need not discourage native educators from developing culturally relevant instruction. Effective teaching of Indigenous students does however require adaptation of teaching pedagogies (Waubageshig 2016).

While Canada performs well among G-7 countries in terms of student outcomes among the bottom-quarter of students ranked by ESCS, the provincial top-to-bottom quarter outcome differences are nonetheless large. Targeted programs addressing bottom-quarter students, such as Pathways and early child education, overlap with programs addressing Indigenous students. These programs can yield benefits. However, provinces must accept the consequences, education budget increases.

Finally, the subject in which Canada’s relative international performance is weakest, and the decline has been largest, is mathematics. Quebec’s superior mathematics performance suggests that other provinces place more emphasis on mathematics in teacher training, and adopt mathematics curricula similar to those of Quebec.

Download the Policy Brief.

¹ The PISA 2018 reports discuss the robust positive relationship between national reading scores and national literacy rate of parents. Parental literacy is measured by the national scores for those in the age 35-54 cohort for the PIACC, an OECD survey of adult core competencies (PISA 2019a, 69).

² Reading, mathematics, and science results were initially benchmarked at an average score of 500 and standard error of 100 for the overall OECD member country survey.

³ At 10 percent significance level, the ninth decline, science in New Brunswick, is also statistically significant.

⁴ The source of results summarized in this paragraph is O’Grady et al. (2019, Tables 1.3,3.4,3.5).

⁵ PISA makes extensive use of a socio-economic-cultural index (ESCS) in attempting to distinguish the relative importance of school and non-school factors. The socio-economic status of a participating student’s family is estimated from several features of the student’s family: parents’ education, parents’ occupations, number of home possessions that can be taken as proxies for material wealth, and the number of books and other educational resources available in the home. The ESCS index of economic, social and cultural status is a composite score derived from these indicators via principal component analysis. It is constructed to be internationally comparable.

⁶ As in Quebec, the decline in mathematics scores in Prince Edward Island between 2003 and 2018 is not statistically significant. However, in the case of PEI, this is largely due to the fact that the standard errors surrounding its provincial estimates are large due to its small population.

⁷ The rationale for not divulging Indigenous results is expressed in a personal email from the CMEC.

Anderson, B., J. Richards. 2016. Students in Jeopardy: An Agenda for Improving Results in Band-Operated Schools. Commentary 444. Toronto: C.D. Howe Institute.

Bennett, P. 2018. What can be Learned from Quebec’s Math Prowess? Policy Options. October.

British Columbia (BC). 2020. “Curriculum Orientation Guide.” Accessed 2020/07/05 at https://curriculum.gov.bc.ca/

Canada. 2019a. Federal Transfers to Provinces and Territories. Department of Finance.

Canada. 2019b. Evaluation of Pathways to Education. Employment Social Development, Canada.

Environics Institute. 2010. “Urban Aboriginal Peoples Study.’

Gang, N., K. Zimmerman. 2000. “Is Child like Parent? Educational Attainment and Ethnic Origin.” Journal of Human Resources 35(3):550-569.

Hanushek, E., M. Piopunik, S. Wiederhold. 2018. “The Value of Smarter Teachers: International Evidence on Teacher Cognitive Skills and Student Performance.” National Bureau of Economic Research Working Paper Series. http://www.nber.org/papers/w20727.

Ludeke, S., M. Gensowski, S. Junge, O. John, S. Andersen. ca.2018. “Does parental education influence child educational outcomes? A developmental analysis in a genetically informative full population sample and adoptee design.” Conference paper available from author.

May, S., A. Jang-Jones, A. McGregor. 2019. PISA 2018: “New Zealand Summary Report, System Performance and Equity.” Ministry of Education.

OECD. 2019a. “PISA2018 Results: What Students Know and Can Do.” Volume I.

OECD 2019b. “PISA2018 Results: Where All Students Can Succeed.” Volume II.

O’Grady, K., K. Fung, P. Brochu, L. Servage, Y. Tao. 2019a. “Pan-Canadian Assessment Program: PCAP 2016, Contextual Report on Student Achievement in Reading.” Council of Ministers of Education, Canada.

O’Grady, K., M. Deussing, T. Scerbina, Y. Tao, K. Fung, V. Elez, J. Monk. 2019b. “Measuring up: Canadian Results of the OECD PISA2018 Study.” Council of Ministers of Education, Canada (CMEC).

Ontario. 2020. “New Math Curriculum for Grades 1 – 8.” Accessed 2020/07/01 at https://www.ontario.ca/page/new-math-curriculum-grades-1-8#section-0

Oreopoulos, P., R. Brown, A. Lavecchia. 2015. “Evaluating Student Performance in Pathways to Education.” E-Brief 203. Toronto: C.D Howe Institute.

Richards, J. 2018. Pursuing Reconciliation: The Case for an Urban Off-Reserve Agenda. Commentary 526. Toronto: C.D. Howe Institute.

_______, J. Hove, K. Afolabi. 2008. Understanding the Aboriginal / non-Aboriginal Gap in Student Performance: Lessons from British Columbia. Commentary 276. Toronto: C.D. Howe Institute.

_______, P. Mahboubi. 2018. “Measuring Student Outcomes: The Case for Identifying Indigenous Students in Canada’s PISA Sample.” E-Brief 272. Toronto: C.D. Howe Institute.

Statistics Canada. 2016a. “2016 Census of Population: Aboriginal Identity (9), Highest Certificate, Diploma or Degree (7), School Attendance (3), Registered or Treaty Indian Status (3), Residence by Aboriginal Geography (10), Age (13A) and Sex (3) for the Population Aged 15 Years and Over in Private Households of Canada, Provinces and Territories.” 2016 Census – 25% Sample Data, cat. no. 98-400-X2016264. Ottawa.

_______. 2016b. “2016 Census of Population: Aboriginal Identity (9), Highest Certificate, Diploma or Degree (15), Labour Force Status (8), Registered or Treaty Indian Status (3), Residence by Aboriginal Geography (10), Age (10) and Sex (3) for the Population Aged 15 Years and Over in Private Households of Canada, Provinces and Territories.” 2016 Census – 25% Sample Data, cat. no. 98-400-X2016266. Ottawa.

_______. 2018. “Education Indicators in Canada: An International Perspective.” https://www150.statcan.gc.ca/n1/pub/81-604-x/2017001/t/tblb1.1.1-eng.htm

Stokes, C. 2008. “The Role of Parental Religiosity in High School Completion.” Sociological Spectrum 28(5):531-555.

Stokke, A. 2015. What to do about Canada’s Declining Math Scores? Commentary 427. Toronto: C.D. Howe Institute.

Thomson, S., L. DeBortoli, C. Underwood, M. Schmid. 2019. “PISA2018: Reporting Australia’s Results. Vol. I, Student Performance.” Australian Council for Educational Research.

Waubageshig [McCue] 2016. “First Nations Elementary-Secondary Education: A National Dilemma.” Verbatim. Toronto: C.D. Howe Institute.

A trained economist, John Richards has written extensively on social policy in Canada, primarily via the C.D. Howe Institute where he holds the Roger Phillips chair in social policy. His current social policy focus is on Aboriginal policy. He is also the co-editor of Inroads, a Canadian policy journal. A professor at Simon Fraser University’s School of Public Policy, Richards has taught and conducted research in Bangladesh over the last decade and heads a modest policy institute linked to the International University of Business Agriculture and Technology.