In complex areas like the study of racial inequality, a fundamentalism has taken hold that discourages sound methodology and the use of reliable evidence about the roots of social problems. We are not talking about mere differences in interpretation of results, which are common. We are talking about mistakes so clear that they should cause research to be seriously questioned or even disregarded (Jindra and Sakamoto, 2023).
In recent years we have heard many allegations of systemic bias and racism in various sectors, including education, health and indeed in the New Zealand workplace. We have also heard allegations of racism within our universities. It is asserted that it is particularly difficult for Māori and Pacific academics to find employment in the universities and, once within the system, to achieve competitive salaries and promotion into the professoriate. Unfortunately, on occasion such claims are advanced in conjunction with personal, sometimes vitriolic, attacks on colleagues and attempts to damage their careers.
Studies of possible systemic bias and racism in academic appointments and promotions are very valuable and may provide insight into possible injustice. For example, when minority staff and students report feelings of isolation, then we must take notice and attempt to address issues that could lead to feelings of isolation. Further, it is quite legitimate to discuss possible broadening of current definitions of research outputs and outcomes to include professional activities of minority researchers that bring benefits to their communities. However, discussions of racism and inequity are highly emotive for many people and there is a danger that both qualitative and statistical studies fail to meet acceptable standards of rigour and objectivity.
In any research study the link must be clear and justifiable between findings, such as inferred differences across groups (e.g. in workplace remuneration and promotion), and the causes advanced by researchers as accounting for those findings. Correlation and causation are not the same notions, but in various papers the two are conflated (e.g. McAllister et al., 2020, titled Glass Ceilings in New Zealand Universities). The authors assert that observed differences in employment outcomes for Māori and Pacific university staff derive necessarily from institutional racism, but fail to consider a wide range of other possible causes.
In that study, age was used as a predictor of remuneration and promotion in New Zealand universities. However, age is not the most appropriate covariate (a covariate is a variable, not of primary interest, but nevertheless of sufficient importance for consideration within a statistical analysis) for New Zealand universities when comparing employment across ethnicities, because typically Māori complete their Ph.Ds and join university faculties at a much greater age than others – up to twenty years later, on average. Controlling for age directly in the relevant statistical modelling, as opposed to using a covariate such as years since completion of the Ph.D, may have led to the estimation of differences in outcomes for Māori and Pacific staff (i.e. provision of salary increases and award of promotions) to be much greater and much more negative than they are in reality. A further limitation on the use of age as a covariate in that study is that age, normally treated as a continuous variable, appears to have been approximated by the decade of birth. Possibly this approach was unavoidable, but the implications should have been discussed.
A more serious problem is the use of population-based denominators (e.g. 16.5% of Māori in the total population) to calculate and compare statistics on representation in the university workforce. Instead, they could have used more appropriate denominators, such as percentages of all Ph.D completions achieved by Māori and Pacific students at least a decade prior to consideration for promotion or years in the academic environment. After that, a perfectly legitimate question is why such small percentages of Ph.Ds are earned by Māori and Pacific students. Finally, possible interventions can be considered.
Studies such as the above could identify injustice and support programmes designed to address injustice, but precisely the same standards of objectivity and robustness of analysis and reporting should be expected of them as in any other area of research. As an example of subjective reporting in the above paper, the lower Performance Based Research Fund (PBRF) scores of Māori and Pacific academics compared to non-Māori/non-Pacific male academics are mentioned, but not discussed. Also given little discussion is the admission that Māori and Pacific do not necessarily prioritise publication.
Any statistical procedure used to model quantitative information must be appropriate, sufficient data of high quality must be available in order to ensure that the models provide useful insight, and the strengths and weaknesses of the procedure made clear. In McAllister et al. (2020), statistical models are applied on data sets that are limited in scope, through no fault on the part of the researchers. However, the limitations of the data and of the models are not discussed. Regression models are used to predict variables, such as the probability of promotion to the professoriate and future earnings, on the basis of only two points in time for about half of the relevant staff, and three points in time for the other half. We are told that the models predict increasing differences over time in probability of promotion between non-Māori/non-Pacific males and other groups, but the data presented in the paper have been aggregated across three funding rounds, rather than presented for each round, and therefore it is not possible to form an independent view on whether differences have increased. Further, the models do not support the assertion of increasing differences clearly because they lack analysis of uncertainty and because they involve an inherently very uncertain process of predicting outcomes into the future. Such models should not be published or taken seriously by other researchers or policy-makers without discussion of uncertainties and detailed commentary on the samples that contributed information to the models.
All research must of necessity embody the strictest possible rigour. Conjecture and motivated reasoning (where emotional bias leads to justification or decision-making) should have no place in any form of research, especially in studies that may influence major systemic changes such as those taking place within health, primary and secondary education, science and research funding today. The suggested causal link between observed differences in employment and remuneration and racism may be true, but remains conjecture until proven true. Nevertheless, this putative link is presented as established fact.
The PBRF is concerned mainly with research, whereas promotion also takes account of teaching and service. I am informed that fewer applications for promotion to Professor-level fail on research grounds than on service and leadership. In fact, almost all applicants have already met research expectations and promotion requires having a sizeable number of graduate students complete successfully, especially Ph.Ds. There is no chance of achieving promotion to the professoriate without a career that is much longer than the PBRF window. Thus, the PBRF and promotions committees measure very different achievements, with little overlap.
The relevant denominator here involves numbers of academics who have graduated with Ph.Ds long enough ago to have a total career that justifies promotion. That is a very different consideration from having had a good recent research career and therefore achieving a high PBRF grade. One can achieve a PBRF grade A in six years, say, but it is impossible to attain Professor-level in six years. Further, ethnicity is not declared in academic promotion exercises and Human Resources administrators are present throughout to ensure that there is no hint of bias. However, having alleged racism in university appointments and promotions, McAllister at al. fail to discuss the academic promotions policies and procedures that have been developed and adopted by our universities.
A previous paper classifies universities as colonial, embodying structural racism that excludes indigenous bodies and ontologies and epistemologies (McAllister et al., 2019). They assert that universities must become decolonised and that indigenous bodies and indigenous ontologies and epistemologies understood as equal partners in creating and sustaining the university. Some very vitriolic and very unfortunate assertions have been made more recently (McAllister, 2022). My own view is that, while there may well be room for improvement in university environments and outcomes for minorities and that research into possible racism and systemic bias is very worthwhile, there is little justification for classifying New Zealand universities as either systemically colonial or racist, and no justification for according equal status to indigenous ontologies and epistemologies to that of world science. On the contrary, the high level of aggressive activism seen today, if unopposed effectively by Government and system leaders, poses a genuine threat to the reputations of institutions and to the reputations and careers of academics who wish to defend science and education against widespread intrusion of traditional knowledge.
In summary, McAllister et al. (2020) fails to meet acceptable benchmarks of research quality on the following grounds:
1. Using inappropriate denominators for calculation of key statistical estimates.
2. Engaging in statistical modelling on constrained data without discussion of the limitations of the data.
3. Failing to present confidence intervals on critical graphs.
4. Failing to discuss unavoidable uncertainties when modelling future estimates from past or current data.
5. Advancing particular, politics-driven causes for unequal representation of ethnicities while failing to discuss alternative causes.
Consequently, allegations made by the authors in relation to racism in university appointments and promotions appear to be unjustified. A more positive approach on the part of the authors might be to suggest the presence of bias and racism and discuss objectively, among other possible causes.
Representation of Māori and Pacific in University Academic Staff
Among many others, one researcher in particular has claimed repeatedly that universities exhibit racism towards Māori and Pacific staff. This person was invited to give plenary talks and the Royal Society Te Apārangi presented an award for her part in a 2019 joint paper "Why isn't my professor Māori?" (McAllister et al., 2019). In the abstract to that paper the authors assert that Māori academics are severely under-represented at universities, comprising approximately 5% of the total academic workforce. In a later paper, McAllister et al. (2020), we read that 8% of Lecturers, 4.7% of Senior Lecturers, 3.4% of Associate Professors and 3.1% of Professors are Māori. Apparently, these statistics average out to approximately 5% overall.
In the 2019 paper the authors go on to state that this situation raises questions about the will of universities to build a sustainable Māori academic workforce, their level of commitment to The Treaty of Waitangi and the associated notion of partnership, and the efficacy of academic equity and diversity policies. Indeed, we must listen when minorities inform us of racism and when women speak of misogyny and bias in science and tertiary education. Certainly, the possibility that these behaviours exist among particular individuals and in some environments cannot be ruled out, and very probably they do indeed exist. In New Zealand today, we also see many initiatives designed to assist minorities, especially Māori, including various financial assistance; scholarships and other education-related incentives; preferential admission to Medical School; heavily Treaty-centric, matauranga Māori-based early childhood, primary and secondary education curricula; an increasingly Treaty-centric tertiary sector; a Treaty-centric public service; naming of public institutions in Te Reo and, of course, a dedicated health authority. Today, the PBRF is undergoing reconfiguration that confers advantage to Māori researchers and Māori research.
In a previous article on workplace bullying (Lillis, 2022), I noted that one form of bullying consists of denying promotions to excellent people. The modestly-qualified, and possibly young and inexperienced, manager continually finds reasons as to why a senior and highly accomplished person does not deserve a small increase in remuneration and enhancement in status. This behaviour does indeed occur and I have recorded it from interviews with senior people, but it is an entirely separate question as to whether particular ethnic groups are denied deserved promotions in our universities on a systematic basis.
A close personal friend and one of the very brightest people known to me (the top graduate across all disciplines at his New Zealand University; a brilliant first-class honours degree in pure mathematics, a Master’s degree with high distinction in pure mathematics, numerous scholarships, including the offer of a Prince of Wales Commonwealth Scholarship to Cambridge University) has informed me of his deflation at repeatedly being denied a minor promotion within the public service after many years of outstanding effort. By chance, the manager who consistently denies the hoped-for promotion was my own manager several years further back in time. I confirm that his treatment of certain staff back then was quite appalling.
The figure of less than 5%, as quoted by McAllister et al. (2019), of university professoriate who are Māori does look small by comparison with the 16.5% of the total population who self-identify as Māori. Without doubt, research in this area should be encouraged in order to identify and highlight possible inequities in the New Zealand workplace. Indeed, I have observed and documented extremely vicious workplace bullying in organisations where I have worked and therefore I am perfectly aware that people can be treated very unfairly (e.g. threats, psychological and physical intimidation, denial of deserved salary increases and promotions, and even lying about a person’s performance).
While racism in relation to employment and promotion within universities is unlikely, racism within certain other organizations and on the part of particular persons is very probable. However, research into difficult areas involving diversity, equity and systemic bias can be influenced by emotions and subjectivity can suffer (e.g. Jindra and Sakamoto, 2023), especially where issues of race and religion are concerned. It is also very critical that quantitative studies are conducted and reported objectively, because inappropriate techniques and faulty analysis can give rise to very misleading conclusions.
In the end, labels such as racism, systemic bias, conscious and unconscious bias and colonialism not only may be applied without much justification, but possibly detract from our efforts to address the real causes.
Use Doctoral Degree Completions as the Comparison
It is indeed tempting to compare the percentages of Māori employed as professoriate staff within New Zealand universities with their 16.5% of the total population and, at face value, achieving proportional representation for staff within our universities may appear to be a laudable target. McAllister et al. (2019 and 2020) do exactly this. McAllister et al. (2019) noted a focus on the part of certain institutions on recruitment and retention of greater numbers of Māori academics so that levels more accurately reflect their proportion of New Zealand’s general adult population. They see increasing Māori staff numbers to be in proportion with the Māori population as a positive step and they argue that, for universities to have meaningful partnerships with Māori, Māori academics should be employed at all levels of seniority and exist in all colleges and departments in universities at greater numbers than population parity. We may agree with efforts to achieve greater minority representation at senior levels in the workplace, but much literature is available on failure of diversity, equity and inclusion programmes in achieving desired outcomes (e.g. Dobbin and Kalev, 2016).
Do the observed low representations of Māori and Pacific staff imply racism? Though racism and systemic bias cannot be ruled out entirely, systemic racism within our universities is highly unlikely because systems are in place to safeguard the decision-making process (e.g. presence of Human Resources staff). Various studies do indeed use the 16.5% population figure for investigations of workplace representations but, since senior academics within the professoriate are drawn from those who hold the Ph.D degree, then a better measure for universities is the percentage of Ph.Ds completed by Māori at least a decade prior to initial appointment. Another measure is years in employment within the academic environment.
The following graph shows Ph.D completions by ethnicity between 1998 and 2021.
To create this graph I have taken raw Education Counts data (Ministry of Education, 2023) and converted numbers to percentages. We see that, over the period from 1998 to 2021, Europeans and Asians obtained Ph.Ds roughly in proportion to their presence in the total population (approximately 70% for European and 15% for Asians, respectively). However, the same is not true for Māori or Pacific, who earned Ph.Ds in smaller percentages than their population statistics would predict.
However, Māori are not badly underrepresented if we consider doctoral completions a decade or so before consideration for promotion, rather than their 16.5% of the total population. The following table gives percentages of all Ph.D completions attained by Māori and Pacific candidates from 1998 to 2021.
1998 - 2008
1998 - 2012
1998 - 2021
The average percentage of all Ph.D completions achieved by Māori candidates over those 24 years was approximately 6.5%. However, the average percentage of all Ph.D completions completed by Māori students over the period between 1998 and 2008 was approximately 4.6%. Thus, the percentages of Māori at Associate Professor-level (3.4%) and Professor-level (3.1%) in 2018 are indeed somewhat, but not greatly, below those expected from Ph.D completions, a decade or more earlier. We see that 8% were Lecturers and 4.7% Senior Lecturers – above the 4.6% of relevant completions.
In evaluating possible underrepresentation of ethnic groups, we should also consider employment of Ph.D-holding Māori and Pacific within the public service and private sector. Clearly, Māori gaining employment outside tertiary institutions diminish numbers seeking employment at the universities. McAllister et al. (2019) note that a large majority of Māori with Ph.Ds are employed outside “the academy”, but also that it is unknown whether this is by choice or because of the lack of post-Ph.D pathways within the academy.
Percentages of Staff at Various Levels within the Universities
I have sourced data on employment at different levels in university faculties from Table 2 of McAllister et al. (2020) in order to create the table below:
These data have been aggregated from each of the 2003, 2012 and 2018 Performance Based Research Fund (PBRF) funding rounds and for those who provided gender and ethnicity information. Other academic levels are not specified in full but could include tutors, tutorial assistants and other assistants. Here, non-Māori/non-Pacific are grouped together as in the original table where, for convenience, I have labelled this group as NonMP. These data are depicted in the graph below.
Clearly, the more senior the level, the fewer Māori and Pacific are to be found at the higher level and, indeed, research is needed in order to identify the reasons. Systemic bias may play a part but is unlikely, and length of service, publication record and other reasons may explain the lower representations at the higher levels.
Limitations of the “Glass Ceilings” Study
The impacts of including staff in the study who “ever” reported Māori and Pacific ethnicity (presumably - even only once) are unknown, but probably are relatively minor and no criticism is implied in relation to this issue. Undoubtedly, an argument can be made for increasing their presence at the most senior levels in order to enhance the experiences of Māori and Pacific students during their years at university. However, if any demographic group takes Ph.Ds in particular areas such as law, business and education but not to the same extent in other disciplines, such as the sciences and engineering then, of course, they cannot be represented significantly in those other faculties. Therefore, it is of great interest to identify the distribution of subjects and topics taken by successful Māori and Pacific doctoral students since 1998. Here we note Kidman and Chu (2017) who reported that very few Māori senior faculty are employed outside of Māori departments.
Evidently, from the graph above, the higher the academic status, the less likely that Māori and Pacific are to be found at that level. Could this be the result of racism? Perhaps so, but other possibilities include the following:
- Differences in teaching performance, publication record and service.
- Māori and Pacific concentrated in a few subject areas, especially in Māori Studies departments.
- Māori and Pacific researchers not necessarily prioritising publications (McAllister et al., 2020).
- Māori and Pacific academics having lower PBRF scores than non-Māori/non-Pacific male academics (McAllister et al., 2020).
- Teaching and administration not accounted for in the PBRF but considered for promotion within universities.
- Māori often beginning their academic careers at much later ages.
- The PBRF and university promotions committees measuring different achievements, with very little overlap.
- Lower earnings for women arising partly from taking time out to raise families.
- Grade inflation for Māori/Pacific at the hands of PBRF panels.
Naturally, later starts in careers mean that, on average, Māori and Pacific have fewer years as senior, experienced researchers and imply less research experience and fewer publications and other outputs.
The authors assert that the PBRF provides a uniform measuring tool for academics in New Zealand and therefore can act as a “magnifying glass” through which to examine the system’s current inequitable treatment of Māori and Pacific academics. While possibly true, the authors appear to presuppose inequity and thereafter proceed to attempt to justify their assertion. In Appendix 1 of this article, I discuss feedback from academics on PBRF ratings. In general, the feedback given to me on those ratings is that they are not robust as measures of performance as assessed in academic promotion exercises.
Comments on the Statistical Aspects of the “Glass Ceilings” Study
McAllister et al. (2020) appear to presume racism and then attempt to justify that assertion – intentionally or otherwise. In addition, as stated earlier, their use of population-based denominators (e.g. 16.5% Māori in the total population) inflates public perceptions of the issue of representation of ethnicities. Instead, they should have used more appropriate denominators, such as percentages of all Ph.D completions achieved by Māori and Pacific students at least a decade prior to consideration for promotion.
The authors control for (i.e. take account of a variable in order to eliminate its effect on other variables) performance, age and field. However, performance is measured by different committees and PBRF scores could be inflated for some groups relative to others. They mention, but do not discuss, the fact that Māori PBRF scores are in fact lower than those of non-Māori males but it is quite possible to have lower, but nevertheless inflated, scores. Very probably, the study suffers from under-controlling and information on other important mediating variables (e.g. field of Ph.D study, publishing in high-impact as opposed to local journals, and prior employment) is not available and cannot enter into the analysis.
In addition to possible under-controlling, age is not an appropriate covariate (a variable, not of primary interest, but nevertheless to be considered within a statistical analysis) because Māori join university faculties at a much greater age than others. A further limitation on the use of age as a covariate is that age, normally a continuous variable, appears to have been approximated by the decade of birth. Possibly this approach was unavoidable, but should have been discussed.
A better covariate is years in the faculty and, perhaps better still, years since completion of the doctorate. In another paper - Why isn't My Professor Maori? (McAllister et al., 2019) - we hear that the average age of the Māori doctoral candidate is 49 years. If they complete their Ph.Ds at 51 or 52 years, say, then that delay must limit their rate of progress by comparison with that of others. The average age for a Ph.D recipient is approximately thirty-one-and-a-half years (Academia Insider, 2023), so that on average Māori gain their Ph.Ds approximately twenty years later. The implication here is that their research careers are also approximately twenty years delayed and possibly about twenty years shorter.
Using age as a covariate has made the various analyses predict substantial differences in treatment of Māori and Pacific staff (i.e. provision of salary increases and award of promotions). As far as the statistical procedures are concerned, all else being equal, a 50 year-old Māori academic in a law, business, medical, science or other faculty should have a roughly similar publication record to a 50 year-old European or Asian. However, often he or she does not because of many fewer years in research and, in some cases, because of not prioritising research, but the statistical model sees the promotions and salaries as more different and, by implication, more unfair across the ethnicities, than in real life. The models will estimate lower p-values than they should for differences in promotions and salaries, and the estimated p-values for certain differences could even become highly significant when in reality they should be non-significant.
The authors control for field (i.e. Business, Science, Medicine, Education, Humanities, Other), but do not specify the range of subjects, nor how many Māori/Pacific and others are located within each field. Thus, it is impossible for the reader to judge the effect of controlling for field on the analyses described in the paper. McAllister et al. (2019) cite Kidman and Chu (2017), who noted that very few Māori senior faculty are employed outside of Māori departments. McAllister et al. believe that this fact is unsurprising, given that Māori departments act as culturally safe spaces within the neoliberal university. Did the authors consider the possibility that personal choice, doctoral degree topic, lower prioritization of publishing of some Māori, late starts to academic careers and finding employment outside the universities could at least partly explain the low level of employment of Māori outside Māori departments?
Controlling for field may or may not impose substantive impacts on model predictions, but we cannot know the impacts without further information on fields, especially on the numbers of academic staff of different ethnicities employed within them and the job performance of Māori and others in those fields. However, that Māori are concentrated mainly within Māori Studies departments is a cause for concern for the robustness of the statistical modelling.
The authors control for PBRF score but it is impossible to know whether scores allocated across subject areas by different evaluation panels are truly comparable. If scores are inflated in one subject area relative to others then, again, p-values for certain estimated differences could become highly significant when in reality they should be non-significant or marginally significant.
Recent Differences in Earnings
In the table below I present adjusted earnings information for 2018, given in Table 3 of McAllister et al. (2020), where Non-Maori/non-Pacific Males serve as the reference group.
Indeed, we see substantial differences in average earnings between Non-Māori/non-Pacific Males and other groups. The authors have made a laudable attempt to be objective about differences in earnings by reporting the downward adjustments arrived at following controlling for field, age and PBRF score and, indeed, significant differences remain after adjusting for those variables. Are these differences inequitable? Possibly so, if Māori and Pacific have equivalent accumulated experience in the academic environment or other similar environments, and if their publication quality and counts are equivalent to those of Non-Māori/non-Pacific. However, on average they do not have equivalent accumulated experience and have not accumulated comparable publication records.
However, as noted previously, years since completion of the Ph.D or else years in academia should have been used as the covariate, rather than age. If these covariates had been used in place of age, then the estimated adjusted differences would have been even smaller. For example, a 55-year old Māori researcher who began her career at 48 years of age might have produced the same number of publications as a 38-year old Asian woman, but the process of controlling for age as a covariate would not take this possibility into account and instead would treat them as equals in their publication records and other achievements.
The following table gives the differences in estimated earnings between Non-Maori/non-Pacific Males as the reference group and the other groups, where the unadjusted differences and the adjusted differences (in parentheses) after controlling for age, PBRF score and field (sourced from Table 3 of the paper).
On page 280 we read that by estimating promotions and earnings at 5, 10, 15 and 20 years post an academic’s first PBRF round, the authors found that these ethnic inequities either persisted or increased respectively, over time. However, neither the raw earnings data above nor the models are clear on this point.
Predictions of Future Earnings and Probabilities of Promotion
In McAllister et al. (2020) approximate earnings were determined on the basis of mean salaries for each position within a given institution in 2012. Then the researchers applied the 2012 salary bands to the 2003 and 2018 populations. They state that differences over time are reflective of the population (e.g. changes in promotions, institutions, academic positions), rather than actual increases in salaries. Earnings were then calculated using salary information to reflect and account for proportions of FTE on a pro-rata basis. Then they “used the earnings variable versus salaries for all analyses.” I am unsure what is meant by the last sentence (possibly “versus” means “instead of”) but the data are inherently uncertain. However, use of proxies for actual salaries may have been unavoidable and no criticism of the researchers is intended here.
In Figure 1 of McAllister et al. (2020) we see that the predicted probabilities of promotion some 5, 10, 15 and 20 years following the first funding round, for those with at least one follow-up funding round, will include many staff who went through only one follow-up. The impacts of inclusion of such people within the model (each of whom provides only two data and therefore straight lines that join them) and the continued use of those inappropriate covariates mentioned above, especially age, should have been discussed. If I am interpreting the description of inclusion of staff on page 275 of the paper precisely, then I believe that we have the following total numbers of staff who engaged in one funding round only, two funding rounds and three funding rounds respectively:
Excluding those who completed one round only, we arrive at the following table of inclusions:
If I have calculated the inclusion data correctly, then nearly 49% of the data used in the modelling derives from researchers who completed only two rounds and therefore provide information for only two points in time. Probably, this situation was unavoidable and no criticism is intended here. Moreover, in some of these cases the two points are six years apart, in others nine years and in yet others 15 years apart, though these unequal intervals should not affect the model estimates appreciably, if at all. Mixed logistic models are used to model the probability of promotion at 5-year intervals, out to 20 years. All predictions are given in Figure 1 (and also in Figure 2) without confidence intervals, but the relevant confidence intervals are likely to be very large, especially for Māori and Pacific, where the numbers of staff are relatively small. Again, the authors control for age inappropriately in the relevant models.
We are told that estimated differences in outcomes between Māori and Pacific academics and non-Māori/non-Pacific men increased over time. Here, we presume that the authors refer solely to the probability of promotion because, as noted earlier, we see no evidence of any increase in differences in earnings in either the raw data or the model of Figure 2. However, those who went through only one follow-up round cannot have contributed information that would predict curvature (i.e. an increase over more than two points in time). Thus, it is not reasonable to assert that the estimated differences in outcomes between Māori and Pacific academics and non-Māori/non-Pacific men increased over time when uncertainties have not been presented and when the final data of the models presented in Figure 1 involve predictions into the future, rather than based on real data. The data presented in Table 5 do include standard error intervals, but those data involved several sources of uncertainty prior to calculation of the standard errors and considerable overlap between the model predictions for each group is very probable.
Because I do not have access to the raw data, I must illustrate the dual problems of modelling on the basis of limited data and not providing confidence intervals by creating and modelling fictitious data. Here is a made-up dataset of salaries for two people, John and Mary, in thousands of dollars. The modelling of McAllister et al. (2020) is based on a combination of two and three years of information, but here I have been generous in giving an additional year (2020) and I provide information for both people in all four years.
Rather than the complex logistic and linear regression models of McAllister et al. (2020), here I fit quadratic curves for the earnings of each of John and Mary, along with 95% confidence intervals, in order to produce the following graphs:
In the first graph (on the left) I have plotted the data in the table above and fitted a quadratic curve, along with 95% confidence intervals. The intervals are large because we are fitting on only four data for each person and because the last two points for John deviate somewhat from a quadratic curve. In the second plot, I have adjusted John’s salary for 2018 from $106k to $104k and for the year 2020 from $110k to $112k, in order to approximate a quadratic more closely.
The uncertainties that arise from modelling on limited data are sufficiently large that, in the first graph, those for Mary are subsumed entirely within those for John. The confidence intervals for John in the second fitted model have contracted markedly, thus illustrating the sensitivity of model fitting procedures to slight variations in the fitted data, especially on small data sets. Nevertheless, in the second graph we see considerable overlap. These confidence intervals arise purely as an artefact of the modelling process. Though it is evident that John has always earned a considerably higher salary then Mary, the models and their associated uncertainties cannot discriminate clearly between their respective salary histories.
McAllister et al. (2020) estimate earnings for one funding round into the future – undoubtedly, an interesting exercise, but one which is inherently very uncertain. To illustrate the additional problem of forecasting into the future, I have plotted the dataset of John’s earnings and used a well-known forecasting method, known as Holt-Winters forecasting, to create a forecast out to the year 2040, along with an 80% prediction interval and a 95% confidence interval. A prediction interval predicts the range in which a future observation will lie, while the confidence interval shows the likely range of values of a statistical parameter, such as a mean or standard deviation.
Here, the forecasts beginning at the first datum beyond the last of the original data (presented as a solid red curve) are plotted as blue points, the 80% prediction interval as a lilac-coloured area, and the 95% confidence interval in grey. Clearly, the forecasts are very uncertain, even when we use a dedicated forecasting method such as Holt-Winters. Thus, the problem of not providing confidence intervals in McAllister et al. (2020) is exacerbated because:
1. They fit their prediction models on the basis of a combination of only two and three points in time.
2. Their salary data are estimated, rather than actual, and are already inherently uncertain.
3. They attempt to extrapolate their models one funding round into the future, a procedure that guarantees even wider confidence intervals.
4. They control for age rather than for years in academia or years since completion of the doctorate and age is estimated very imprecisely by decade of birth.
5. They cannot control for covariates that were not collected - through no fault of their own.
Clearly, the authors were aware of the need for successful estimation of parameters and we are told that Māori and Pacific categories were combined in order to ensure that the models had sufficient numbers to promote successful estimation of all parameters. However, because of the absence of confidence intervals in their figures, the reader is unable to ascertain the quality of those estimates and whether or not significant differences in the models in fact exist across groups. We are also told that covariate interactions were considered in the model structure but were excluded because very few were significant. Again, the authors were fully aware of the need to investigate both a range of covariates and interactions, but covariates such as years since completion of the Ph.D or years in the academic environment do not appear to have been collected.
In Figure 2 of McAllister et al.(2020), as before, we see that predicted earnings some 5, 10, 15 and 20 years following the first funding round, for those with at least one follow-up funding round, will also include nearly half with only one follow-up (i.e. only two points in time). Again, the predictions are presented without confidence intervals. Once more, age is used inappropriately in the relevant models. It must be emphasised that in both Figures 1 and 2 of their paper the absence of confidence intervals renders it unclear as to whether or not the model predictions across ethnicities and genders are significantly different.
Several typographic errors are evident, indicating that the paper needed a thorough edit before submission. For example, on page 276 we read: “age (decile of birth)”, rather than “age (decade of birth)”. On page 278 the caption for Table 4 suggests that the table presents odds whereas it should have indicated odds ratios. On page 280 the caption for Figure 2 says “Predicted probability of earnings . . .", when it should have said “Predicted earnings . . . ". Peer-review should have identified such errors prior to publication.
On page 280 we read:
Furthermore, by estimating promotions and earnings at 5, 10, 15 and 20 years post an academic’s first PBRF round, we found that these ethnic inequities either persisted or increased respectively, over time.
Ethnic inequities in earnings may have persisted or increased over time but, as noted above, neither the raw data above nor the models (the model of Figure 2) are clear on this point. Any modelling exercise that both models past outcomes and predicts future outcomes should include discussion of inevitable uncertainties, of the samples that contributed to the stated findings and of the procedures used to predict future outcomes.
Finally, in their abstract, they state that in 2018 Māori and Pacific female academics earned on average $7,713 less than non-Māori/non-Pacific male academics and had 65% lower odds of being promoted into the professoriate from 2003 to 2018. Possibly, in the case of Māori and Pacific women, the near twenty-year delay and time out for raising families contribute to the estimated difference in earnings. This possibility raises genuine questions about how to provide for female university staff who raise families.
To summarise – the modelling exercises of Figures 1 and 2 of McAllister et al. (2020) are interesting and worthwhile as in-house investigations, but are inherently highly uncertain and should not have been published without detailed discussion of the limitations of the data and of the modelling procedures.
Why isn’t my Professor Māori?
In the joint paper of the above title (McAllister et. al., 2019) the authors refer to renewed calls for universities to be held accountable for their failure to address institutional racism in the ranks of the professoriate. Here, the authors once more presuppose an outcome (the presence of racism) and only after that assert its truth. We can certainly agree, however, with the notion that growth in numbers of Māori students impacts upon current Māori staff workloads if similar increases in the number of Māori academics to support Māori students do not occur. They state:
This finding raises questions about the will of institutions to build a sustainable Māori academic workforce, their level of commitment to Te Tiriti o Waitangi and the notion of partnership, and the efficacy of academic equity and diversity policies.
and . . .
We argue that the gap between the values espoused in university equity policies relating to Indigenous peoples and the starker reality of academic hiring practices - which based on our data and previous studies appear to favour European scholars - needs to be registered and openly discussed if decolonising change is to take place.
This is where the authors concede that Māori often enter their academic careers later, and that the average age of Māori doctoral candidates is approximately 49 years. They concede that, while the late start probably contributes to the lack of Māori at senior levels, other factors, including a lack of institutional will to appoint Māori staff in the upper echelons of the academic workforce or the absence of effective institutional strategies for the hiring and promotion of Māori academic staff, are likely to be significant. This assertion might be true but for now remains pure conjecture. In Appendix 2, I discuss possible assessment of the wider impacts of research and that could, indeed, recognise other contributions of Māori and Pacific more than at present.
What is meant by decolonising change and why does the notion of partnership arise for one demographic group when it is not appropriate for others? Is partnership mentioned in the Treaty of Waitangi? In Appendix 3, I discuss further the problem of lack of objectivity that is evident in some ideologically-driven research.
Possibly, partly as a result of articles such as McAllister et al. (2020), Government proposes to favour Māori and Pacific research and researchers within the PBRF (Lillis, 2023). Formulae used to allocate money to research organisations through this fund involve numeric weightings that are about to increase substantially for Māori researchers, Māori-oriented research and Māori postgraduate degree completions, and similarly for Pacific. For example, the Subject Area known as Māori Knowledge and Development will have the highest weighting, ahead of engineering and technology; agriculture and other applied biological sciences; architecture, design, planning, surveying; biomedical; clinical medicine; pharmacy; public health; veterinary studies and large animal science; dentistry; and Pacific research and many others. Indeed, a recent proposal for a Faculty of Science paper on mātauranga Māori at a major New Zealand university includes the phrase: Mātauranga is central to the future practice of science in Aotearoa New Zealand.
The benefits of enhanced funding to Māori researchers may indeed include increases in Māori and Pacific employment in tertiary organisations and restoration of mana, but New Zealand cannot afford to fund large volumes of such research at the expense of other research of potential benefit on a system-wide basis. In Appendix 4, I discuss further the question as to whether the observed inequities are fair. In Appendix 5, I discuss other possible reasons for gaps in employment.
Feedback from Academics on Racism in University Promotions
I have inquired about possible racism and discrimination against Māori and other minorities with several contacts of mine at New Zealand universities. They are unanimous in that no such bias exists and that it is nearly impossible for it to occur in the present system of academic promotions. However, they do believe that there is a danger in Māori/Pacific outputs assessed exclusively by Māori/Pacific panels. I am told that ethnicity is not declared in academic promotion exercises and that Human Resources administrators are present throughout to ensure that there is no hint of bias. It appears to be very unlikely that promotion committees could enforce a bias against Māori, even if they wanted to. Conversely, ethnicity is declared within PBRF assessments and there are no protections in the process; for example, no Human Resources staff are involved in order to ensure fairness. If institutional racism were present, it would be very evident in the PBRF.
One very senior university person has stated that the discrepancy between PBRF performance and earnings is not that Māori were deliberately paid less as academics, but that Māori PBRF scores were inflated by certain panels. In other words, there was unofficial positive discrimination or affirmative action. Panels would ignore the assessment criteria and give credit for “doing a lot for their communities" or "providing help to Māori students", etc.
I quote another senior academic:
Those recent complaints on this are no more than exercises in motivational reasoning. They begin with the assumption that discrepancy equals bias, and work backwards from there. University leaders should have defended themselves over these false accusations, but instead they chose to hide until the dust settled. There are a few of these papers but they all have the same flawed assumptions and they all cite each other, giving the impression that the work is highly cited.
Several senior scientists and research leaders have reported that they are unaware of racism in science and that New Zealand universities are keen to enrol minority students in their degree programmes and to appoint minority researchers to academic roles.
Investigation of possible racism and systemic bias within our universities is highly valuable but must remain objective and free of conjecture. Certain studies fall short on both objectivity and robustness and further research is required in order to establish whether or not racism truly exists in university appointments and remuneration. At present, it seems very improbable that systemic racism accounts for differences across ethnic groups in remuneration, appointment to academic positions or in promotion to the professoriate.
Academia Insider (2023). What is the PhD student average age? Too late for your doctorate?
Dobbin, F., and Kalev, A. Harvard Business Review (2016). Why Diversity Programs Fail. https://hbr.org/2016/07/why-diversity-programs-fail
Jindra, M., and Sakamoto, A. (2023). When Ideology Drives Social Science. The Chronicle of Higher Education.
Kidman, J., and Chu, C. (2017). Scholar outsiders in the neoliberal university: Transgressive academic labour in the whitestream. New Zealand Journal of Educational Studies, 52(1), 7–19. http://doi. org/c83f
Lillis, D. (2022). Sorry - but “nga Mihi” isn’t good enough!
Lillis, D. (2023). Capture of Research Funding in New Zealand?
Marie, D., Fergusson, D. M., and Boden, J. M. (2008). Educational Achievement in Maori: The Roles of Cultural Identity and Social Disadvantage. Australian Journal of Education. 52: 2, 183-196. Article first published online: August 1, 2008; Issue published August 1, 2008.
McAllister, R., Kidman, J., Rowley, O., and Theodore, R. F. (2019). Why isn’t my Professor Māori ? MAI Journal. DOI: 10.20507/MAIJournal.2019.8.2.10. p 235-249.
McAllister, R., Kokaua, J., Naepi, S., Kidman, L., and Theodore, R. F. (2020). Glass Ceilings in New Zealand Universities . MAI Journal. DOI: 10.20507/MAIJournal.2020.9.3.8, p 272-285.
McAllister, R. (2022). 50 reasons why there are no Māori in your science department
Journal of Global Indigeneity, Vol.6, Issue 2, 2022, 1-10.
Ministry of Education (1997). Chapple, S., Jefferies, R., and Walker, R.
Ministry of Education (2023). Tertiary Research.
Ngata, T. (2021). Defence of Colonial Racism.
Whitehead, M. (1992). The concepts and principles of equity and health. International Journal of Health Services, 22(3), 429–445. https://doi.org/fwhz5n
Appendix 1: The PBRF as a Proxy for Performance
Not having engaged directly with the PBRF I have no independent view of the merits of PBRF ratings as proxies for performance required for promotion within university faculties. However, my discussions with senior staff reveal a commonly-held perspective that PBRF ratings are badly flawed.
University staff remind us that the PBRF is a funding mechanism for institutions rather than individuals. It does measure recent research contributions of individuals but promotion takes a much wider and longer-term view of an academic's career. Academics are told expressly not to include their PBRF rating in their promotion applications. When they do so anyway, promotions committees ignore it.
Promotion is explicitly whole-of-career, whereas the PBRF is solely about the window since the last round, usually about six years. The PBRF focuses entirely on a small number of nominated outputs, currently four, whereas promotion, especially to Associate Professor and Professor-level, requires a substantial total number of publications. For Natural Sciences at the University of Auckland, for example, the expectation is 80 publications.
PBRF is only about research, whereas promotion takes account of teaching and service as well. I am informed that fewer applications for promotion to Professor-level fail on research grounds than on service and leadership. In fact, almost all applicants have already met research expectations and promotion requires having a sizeable number of graduate students complete successfully, especially Ph.Ds. There is no chance of having achieved this without a career that is much longer than the PBRF window. Thus, the PBRF and promotions committees measure very different achievements, with very little overlap.
Someone who has had a good recent run of research, with a lot of impact, but engaged in no teaching or service, might well receive the top PBRF grade of A, but would have absolutely no chance of promotion to Professor or Associate Professor, and little chance of Senior Lecturer over the bar. Conversely, someone who has had a long, steady research career, graduating plenty of students, reaching the publication total with strong impact, and has contributed strongly to teaching and service, might well get promoted to Professor without ever having been a PBRF grade A.
Finally, there may be a higher proportion of Māori PBRF A grades than Māori professors because of time since appointment. The relevant denominator here is academics who have graduated with Ph.Ds long enough ago to have a total career that justifies promotion. That is a very different thing from having had a good recent research career and therefore achieving a PBRF A. One can achieve a PBRF A in six years but it is impossible to attain Professor-level in six years.
Appendix 2: Assessing the Wider Impact of Research?
McAllister et al. (2020) note that the PBRF does not effectively assess the wide impact that Māori scholars have beyond simply publishing in high impact factor journals, indicating that not all of the work that Māori scholars do is counted towards PBRF, thus making invisible their wider contribution to the research sphere. Perhaps we can agree that contributions other than journal publications should be taken into account within the PBRF and other funds. They assert, as did others prior to their paper, that the system privileges westernised educational approaches and western ideas of research excellence. Of course, the PBRF system is consistent with notions of excellence in world science and research, rather than with purely western notions.
Others have made similar claims to those of the researcher. For example, Tina Ngata has claimed that the famous Letter to the Listener was a true testament to how racism is harboured and fostered within New Zealand academia, as part of a global system that harbours and fosters racism (Ngata, 2021). Again, we acknowledge such assertions, but the truth or otherwise of systemic racism or sexism in science is difficult to determine objectively.
McAllister sees the letter as expressing the authors’ outrage and perhaps their white fragility about Government’s suggested changes to the New Zealand’s National Certificates of Educational Achievement (NCEA) curriculum (McAllister, 2022). She regards the letter as an ill-informed letter attempted to “defend” science from suggested parity with mātauranga Māori, a blatantly racist attack on mātauranga and an attack on her very existence. An alternative perspective is that the authors advanced their considered professional opinions in good faith and at no point engaged in racism or unfair attacks on traditional knowledge.
Appendix 3: Ideologically-driven Studies
In McAllister et al. (2020) we are informed that their study will only add to the growing evidence base that exposes institutional racism within New Zealand universities. It most certainly will not. We are also informed that qualitative research is required to investigate the experiences of Māori and Pacific academics within the promotion system and to understand how racism in academic promotions can be addressed. Most senior New Zealand academics known to me disagree that systemic racism is present in the first place.
We are told of a need for qualitative research that explores how and why universities in New Zealand have continued to uphold practices and habits that discriminate against Māori and Pacific. Have they discriminated and, if so, have McAllister et al. proved it?
In debates on racism and systemic bias, we must have clarity on the difference between equality and equity. Equality refers to equal treatment, irrespective of group membership. Conversely, we might agree that equity refers to representation in organizations in proportion to presence in the total population. Thus, we may have equal opportunity but inequities arise nevertheless because of various causes, including historic discrimination, socialization, cultural differences and biological differences. McAllister et al. (2020) appear to agree with the definition of Whitehead - that inequities are differences in outcomes that are not only unnecessary and avoidable but also unfair and unjust (Whitehead, 1992). However, it is not necessarily true that inequities result from unequal treatment in the present; i.e. structural racism or sexism.
Jindra and Sakamoto (2023) remind us that ideologically-driven abuse of statistics is evident across the social sciences. They believe that in left-leaning academic environments we see biases toward structural causes, which they say occur partly because scholars face pressures to avoid blaming people and cultures for social problems. They assert that social theory should recognize both structure and agency, in addition to intermediary forces of social influence, such as culture.
They state that much of higher education is devoted to explaining inequality by highlighting malevolent structural forces. However, social-class influences are stronger than racial influences on education, income, and health, but strong cultural patterns have emerged in some contexts. They refer to clear errors, or at least to very poor scholarship, that should not have passed peer review. In the sensitive area of racial disparities, simplistic mono-causal explanations, such as structure, should be avoided in favour of multi-causal arguments that involve the mutual influence of structure, culture and individual variation.
They suggest that some of these articles should be candidates for retraction but that retraction is rare. Certain scholars have even received major promotions, perhaps partly because their findings fit particular favoured narratives. Instead, papers that violate ideological beliefs, more than those with errors of fact, receive pressure for cancellation.
Appendix 4: Are Inequities Truly Unfair and Unjust?
McAllister et al. (2020) found that Māori and Pacific male and female academics had significantly lower odds (a statistical measure, closely related to the notion of probability) of being an associate professor or professor or of being promoted, had lower earnings than non-Māori/non-Pacific male academics, and that the observed differences remained after taking account of research performance as measured by PBRF scores, age and academic field. These differences were particularly marked for women. The odds (in favour) of a particular outcome is the ratio of the probability of that outcome to the probability that the outcome does not occur. On the other hand, the odds ratio represents the odds that an outcome will occur, given a particular exposure (e.g. racism in awarding promotions and raising salaries), compared to the odds of the outcome occurring in the absence of that exposure. I have already commented on the inappropriate use of age as a covariate.
They suggested that their findings provide quantitative evidence that Māori and Pacific face racism and that equity initiatives must address institutional racial discrimination and take account of intersectional identities. They refer to institutional racial discrimination and inequitable outcomes for Māori and Pacific academics as unfair and unjust. However, is institutional racial discrimination in our universities real and are those inequities truly unfair and unjust and, if so, necessarily an outcome of racism and bias? Their suggested causal link between observed differences in employment and remuneration and racism is pure conjecture. I quote that paper again:
The government also has an important role to play in creating incentives for universities to address institutional discrimination and to meet Te Tiriti o Waitangi and equity obligations.
Institutional discrimination is again offered as truth. In addition, they assert that further research is required in order to understand how racism in academic promotions can be addressed and to explore how and why universities in New Zealand have continued to uphold practices and habits that discriminate against Māori and Pacific. Here, the authors also presuppose racism. Again I quote:
Moving forward, universities need to take significant steps to actively address institutional racism and sexism in both recruitment and promotion processes. Universities need to urgently reimagine and recreate the promotional system and move towards a system that meaningfully responds to and incorporates Tiriti rights, reflects a Tiriti partnership and recognises Māori and Pacific excellence.
Of course, as discussed earlier in this article, other factors that may affect promotion outcomes include years in the profession, publication record, differences in teaching performance and differences in service. Their paper also notes that Māori and Pacific researchers do not necessarily prioritise publications. Unfortunately, publications are the most commonly-accepted outputs of research internationally, and failure to publish sufficient volumes of work in reputable journals would lead inevitably to very slow rise through the ranks of nearly every university in the world.
Appendix 5: Other Reasons for Gaps in Employment?
Are the existing gaps in employment between Māori and others in tertiary education and research truly critical and do they result from racism? Could they instead constitute a relatively minor issue that may have emerged partly as a matter of the causes mentioned above, personal choice and partly as a result of underachievement in secondary education (itself most probably a consequence of socioeconomic factors and other employment opportunities for Māori Bachelors and Masters graduates)? Surely, decisions made by young people, including Māori, in relation to tertiary study and careers, are indeed largely a matter of personal choice and, in general, appointments to academic positions are made on merit. As a former schoolteacher and tertiary lecturer, I have known many Māori students who hoped to take degrees in economics, business, law and medicine, rather than science, usually or always as a matter of personal choice, as far as I was aware.
It is an easy recourse to ascribe disparities in outcomes in domains such as health, education and employment in science to systemic bias and racism, but possibly the significant causes lie outside the jurisdictions of health, education and science – principally socioeconomic factors. Here we may have a problem of attribution. Claims of bias and racism may be true but are presented without evidence and are difficult to evaluate objectively.
Already, many years ago, research suggested that socioeconomics and out-of-school factors predominate in explaining underperformance of Māori in secondary education. A Ministry of Education report of a quarter of a century ago already gives some insight here (Ministry of Education, 1997):
. . . research shows that Māori students do worse at school than non-Māori students mainly because Māori parents have less money and less education than non-Māori parents. So the gap begins at birth. We think that about two-thirds of the education gap is because so many Māori families have fewer resources. What about the other third?
Māori children tend to go to schools which have many children from families with fewer resources. There is evidence that children do less well at these schools. The research suggests that the other main reason for the education gap, apart from family resources, may be a combination of: - barriers at school - the negative way in which older Māori students, especially boys, react to school. This is partly because of their past experiences of not doing well in the education system.
The research done so far shows that Māori students probably do face some barriers at school. For example: - some students make racist comments - some non-Māori teachers may have difficulty understanding Māori children - there are only a small number of Māori teachers.
More recent research confirms the Ministry of Education view. For example, Marie et al. (2008) found that children of Māori ethnic identification were exposed to significantly greater levels of socio-economic disadvantage in childhood and that control for socio-economic factors largely reduced associations between cultural identity and educational outcomes to statistical non-significance. Their findings also suggested that educational underachievement among Māori can be largely explained by disparities in socio-economic status during childhood.
It is essential that resources are allocated on the basis of correct identification of where in the education system and the wider society the problems occur, rather than to throw money at purported solutions downstream of the problems. It stands to reason that students who perform poorly at secondary level are not as likely to achieve a Ph.D as those who perform well at secondary level. Consequently, fewer of them proceed to take postgraduate degrees and have academic careers. If a remedy exists, then surely it is to be found prior to university study.
Dr David Lillis trained in physics and mathematics at Victoria University and Curtin University in Perth, working as a teacher, researcher, statistician and lecturer for most of his career. He has published many articles and scientific papers, as well as a book on graphing and statistics.