We need to address gender bias in medical research peer review: Jess Borger

Dr Jess Borger, CCS Gender Equity
Diversity and Inclusion Committee

Reproduced from Women's Agenda 28 February 2022

Having explored data on whether Australia’s largest medical research funding body was doing enough to retain women in STEMM on Women’s Agenda (with figures later picked up by Nature), Dr Jessica Borger was recently asked to present on the topic of “breaking the bias”.

Jessica shares the below piece based on the presentation, highlighting opportunities for Australia to lead on opportunities on achieving equality in medical research.

Australia as a country is investing money, time and effort to recruit and train women in science technology engineering maths and medicine (STEMM), but we are dismally failing to retain women in the sector, particularly senior leaders.

Despite an overhaul of Australia’s medical research funding schemes in 2018, implemented in part to improve gender equity, a recent communique by the CEO of the National Health and Medical Research Council (NHMRC) Professor Anne Kelso identified that “women scientists in Australia are less likely to win a major medical-research grant than their male counterparts”.

At the most senior academic level of funding in the newly created Investigator grants scheme, only 21% of awardees were women, equating to an additional $66 million of funding allocated to support the careers of senior male researchers.

This is a disappointing outcome for the Australian Government’s largest medical research funding body, leaving many to question why women and their research continue to be lost from the system.

Remaining conscious of unconscious bias

Peer review is a critical evaluation process to validate many of the key metrics of academic excellence including the award of grants and fellowships. Metrics in which only scientific rigour is considered, such as research publications, have not overwhelmingly been shown to display gender bias. However, grant peer review is unblinded and judges the researcher’s academic profile and leadership, paving the way for reviewer biases to take effect.

Reviewers’ decisions may be influenced by systemic biases based on the information available to the reviewer, such as the applicant’s race or their affiliated institution (Table 1). Reviewers individual biases may compound their decisions by favouring male researcher applications over those of female researchers, even implicitly so. As the Matilda effect captures that there is a clearly established tendency to undervalue women’s scientific achievements it must be assumed that gendered stereotypes will impact some peer evaluations of grants.

Table 1. Implicit bias exists within the peer review process

Today, it is indisputable that many of the academic excellence metrics to which researchers are held to during peer review are a more challenging reality for women (Fig 1). Importantly, these metrics are not independent of each other and contribute cumulatively throughout a researcher’s career to create a leadership profile, placing women at a significant and escalating disadvantage.

Fig 1. Key academic metrics where women experience cumulative bias. Of the few successful senior women retained to compete against male counterparts, they are doing so with significantly reduced funds, limiting their future research pathway for example; reducing the number of people they employ which restricts productivity and delays experimental outcomes, leading to lack of conference invitations and fewer publications culminating in low perceived scientific contributions, cumulatively resulting in the award of less research funding, fuelling the vicious circle. The leaky pipeline occurs at each of these hurdles, forcing women researchers to leave science at early stages of their careers.

Men receive more grants and more funding

The net outcome results in a loss of senior women leaders (Fig 2a).

Strikingly, the NHMRC Investigator Grant outcomes reflect this leaking pipeline, with success rates plummeting with seniority (Fig 2b); only 19% of women received funding at the most senior leadership level (L3). The associated financial fall-out for women medical researchers follows a similar downward spiral from the L1 stage (Fig 2c), resulting in the most senior men being disproportionately awarded a staggering $205 million in the first 3 years of the scheme.

Fig 2. As seniority increases, women receive less grants and funding, aligning with the loss of women in STEMM.

Gender biased peer review

The cumulative, biased impact of peer review overwhelmingly contributes to the significant loss of women in STEMM, resulting in a shortage of women eligible to apply for grants and fellowships at senior levels. Indeed, Professor Anne Kelso claimed the ‘single biggest factor contributing to the greater total value of Investigator Grants awarded to men than women is the higher proportion of male applicants at the most senior levels of the scheme”.

It is striking, that at these senior leadership levels (L1-3) where men receive 20% more funding than women (Fig 3a-b), a matched 20% inequity in the gender of grant reviewers in favour of men is evident. This is in stark contrast to early career researchers (EL1-2, emerging leaders) who are assessed by a gender equitable review process and receive equal funding (Fig 3c-d).

Fig 3. Gendered peer review aligns with gendered funding outcomes.

The outcome of peer review is reflected in the applicant’s average final score, with mean scores reported by the NHMRC as equitable across all leadership levels (Fig 4a). Yet structural priority funding, awarded to ‘near-miss’ grants that score below the cut-off, supported 23% of all successful applications by women revealing that women score significantly lower than men. At the critical L1 stage, one-third of grants to women were scored by gendered reviewers below the cut-off (Fig 4b).

Fig 4. Mean average scores were reported by the NHMRC as similar across genders, yet 23% of successful women received near-miss funding with scores below the cut-off.

The recent 2021 Investigator Grants outcomes show that after removing structural priority data, actual funding rates for women were ~2–4% lower than those for men across both major NHMRC funding schemes. Interestingly, rigorous mathematical modelling has shown bias that skews scores by just 3% can result in noticeable disparities in funding rates.

Strikingly, at the critical L1 drop-off point, where one-third of women receive structural priority funding, the majority of men are awarded at the highest funding tiers (Fig 4). Interestingly,  a condition of structural priority funding is that the funding package is limited to the lowest tier, automatically disadvantaging 23% of successful women. With more men than women receiving funding in the top bracket, women on average received $500,000 less per grant than the men despite being at the same level of seniority. Ultimately, this means of the few successful senior women retained to do competitive research, are doing so with significantly reduced funds compared to their male counterparts, limiting their future research pathway and forcing women researchers to leave science at early stages of their career (Fig 1).

Fig 5. Funding tiers awarded to women are heavily skewed to the lower tiers, resulting in less funding at the same level of seniority.

Addressing the elephant in the room

There is a relative reluctance amongst male researchers to accept evidence of gender biases in STEMM, especially when considering bias impact on a woman’s academic track record during peer-review. By arguing against the existence of bias in peer review and claiming that the focus should be on “funding science on the basis of the scientist work, not their identity”, male peers are unconsciously advocating against women. In a grant scheme where both the research and the leadership of the researcher are considered, due to compounding gendered systemic biases, sadly, identity is the elephant in the room to be addressed.

Emerging women leaders perform equally in terms of metrics and funding, until they move into leadership positions where there is a loss of opportunities and participation by women. Although structural priority funding continues to be in essence a life-line that rescues over 20% of women from the leaky pipeline, why women are scoring lower at peer review needs to be exposed and addressed at a societal, institutional and government funding body level.

Calls for action to fund women in STEMM equitably in a recent petition and position paper submitted to the NHMRC, highlight the need to urgently reintroduce peer review panels to transparently and equitably discuss reviewer assessments of applicants, increasing peer reviewer accountability. Ensuring equal representation of men and women, that have participated in unconscious (and conscious) bias training, will further expedite a fairer grant review process.

For the career track records of women to be considered equal to men, structural changes need to be implemented within funding agencies. Reforms including reviewing men and women in separate pools and allocating equal amounts of funding to each gender, would enable equal representation and retention of women leaders in senior positions that men have dominated.

With the emergence of other countries applying gender quotas for research and gender quotas in female leadership, instead of being left behind, Australia currently has a chance to be leaders in attaining equality in medical research.