‘Golden tickets’ on the cards for NSF grant reviewers

The US National Science Foundation (NSF) is considering piloting a funding model under which grant peer reviewers are given ‘golden tickets’ — a veto that they can deploy just once in a funding cycle to override the decisions of their colleagues. Peer reviewers often disagree on what grants should be funded. The golden-ticket model would, in theory, mean that unorthodox, high-risk proposals that don’t necessarily have the unanimous backing of all referees end up being funded.

“I’m sure every researcher in the world feels like their unfunded proposals were innovative and outside of the box and the reviewers just didn’t get it,” says Karl Zelik, a mechanical engineer at Vanderbilt University in Nashville, Tennessee.

Pushing boundaries

Zelik sees golden tickets as an opportunity for applicants who want to propose ideas at the fringes of their field that are unlikely to convince all of the researchers on a review panel. “I think there are many researchers who look back on their career and feel like the best work they’ve done was also some of the most contentious”, but which at the time “was repeatedly rejected”, he says.

Canada announces new innovation agency — and it’s not modelled on DARPA

Some funding bodies in Europe have already implemented golden-ticket schemes. In 2017, the Villum Foundation, one of the largest philanthropic funders in Denmark, committed to allocating 15% of its annual funding to ‘risky’ grants that applicants might not otherwise consider submitting. In 2022, 51 researchers received 100 million kroner (US$14.3 million) through the scheme.

The Volkswagen Foundation in Hanover, Germany, ran a similar scheme from 2013 to 2021, but restricted the use of golden tickets to grant-review meetings, and excluded written assessments. Each reviewer was allowed to use the golden ticket once per meeting. During the time period, only 11 of 183 grants awarded were on the basis of a golden ticket, a spokesperson for the Volkswagen Foundation says. “One important effect of the golden-ticket option was to save time in the meetings, when two opposing arguments or opinions could not be resolved by further deliberation.”

Finding excellence

“The peer-review system was designed to assess what is good science and what is bad science, and it is really good at that,” says Thomas Sinkjær, a medical researcher at Aalborg University in Denmark who was involved in testing golden tickets in 2016–17 when he was the director of the Villum Foundation in Søborg. “When it comes to [choosing] between very, very good proposals and extremely excellent proposals, it’s not very good.” In these cases, he says, reviewers’ biases — such as those based on an applicant’s gender, career stage or institution — might kick in and influence the final decision.

The evidence suggests golden tickets “could be a mechanism to fund high-risk, high-reward research”, says Erwin Gianchandani, assistant director for technology, innovation and partnerships at the NSF in Alexandria, Virginia. That’s why the NSF is considering a pilot scheme of its own, but details of how the agency would run such a scheme remain scarce, Gianchandani explains. “We are studying what’s out there and trying to learn lessons from any [relevant] literature.”

Golden tickets are not without their naysayers, however. Critics worry that the system could be gamed by rogue reviewers, who use their tickets to fund friends or colleagues. To weed out that problem, the Villum Foundation blinded its grant funding process, making it harder for referees to discriminate between grants listing certain authors and more difficult for their implicit and explicit biases to creep in, Sinkjær says.

A level playing field?

The Villum Foundation also made sure that reviewer panels don’t meet to discuss proposals, Sinkjær notes. After rolling out the golden-ticket scheme, the foundation found that there was no correlation between success in securing grants and the career stage of applicants. “You’re as successful as a postdoc when applying to this programme as you are as a full professor,” Sinkjær notes.

Billion-dollar US health agency gets new chief — but its direction remains in limbo

But Heidi Williams, an economist at Stanford University in California who specializes in science policy, says that blinding proposals could exacerbate disparities in science. She notes, for instance, that experienced academics might write better grant proposals than do early-career colleagues, and that blinding the reviewing process means researchers from marginalized groups can’t be given extra support.

Golden tickets are the latest in a national trend in the United States to promote high risk, high-reward science. In March last year, the Biden administration founded a billion-dollar federal funding agency, the Advanced Research Projects Agency for Health (ARPA-H), targeted specifically at finding biomedical innovations. The agency is the latest of its kind after DARPA, which targets defence research; IARPA, aimed at the intelligence research community; and ARPA-E, which focuses on developing new energy technologies.

“If what you want to go for is really truly innovative research ideas,” Sinkjær adds, “you will really need to have the focus on the research idea and not primarily on the person proposing it.”