Does Massive Research Funding Work?: Lessons from South Africa

Does research output increase significantly when funding allocation is concentrated on a small number of elite researchers?

We examined this question by exploiting the design features of funding mechanisms used by the South African National Research Foundation (NRF). The NRF allocates large funding ($150,000 – $300,000 per annum for a minimum of 5 years, renewable to a maximum of 15 years) to 80 research chair holders selected by the NRF as world-class in their fields. The introduction of this mechanism in 2008 identifies a discrete time point after which the performance of the research chair holders can be contrasted with the performance of comparable researchers who did not receive the funding allocation associated with the chairs.

Appropriate control groups can be constructed for the South African context by means of at least two methodologies.  First, the NRF itself operates a rating system of researchers which ranks researchers in a set of categorical tiers based on peer review of the quality of their output, which is independent of the research chair selection mechanism. While the NRF publishes an array of ratings, the A and B rating categories are those that are held to indicate world class research output, and offer an immediate control group against which to compare the performance of the research chairs. Importantly, the research chairs are themselves subject to this peer review rating system. Second, we also identify a control group based on propensity scores loading on objective bibliometric performance measures of researchers.

Our principal findings are as follows.

First, for the 2009-12 period there is no statistical difference between the performance of the NRF chairs on average and A-rated researchers without chair funding, nor between the average NRF chair and researchers without chair funding who under the propensity score matching methodology had the strongest pre-award research performance. Despite a minimum 15:1 funding advantage, there is thus no statistically observable superior research performance on the part of the NRF research chairs.

Second, there is a strong dispersion of performance amongst NRF research chairs. The stronger the research performance of research chair recipients prior to the award of the funding, the stronger the subsequent improvement in performance, under both the NRF peer review and propensity score matching methodologies. By contrast, research chair recipients with poor prior research records performed worse than the control groups despite the receipt of the substantial research funds. Strikingly, more than 50% of the South African research chair recipients had a NRF rating that indicates no international peer recognition. Symmetrically, under the propensity score matching methodology the probability of receiving a research chair is maximized amongst researchers with the lowest research performance prior to the award. The peer-based selection of the research chairs thus appears to have been significantly biased away from the stated goal of selecting research excellence.

Third, funding shows a differential rate of return across disciplines. Consistent statistically significant returns are evident only with respect to the Biological, Medical and Physical sciences, only weakly so for Chemical and Engineering sciences, and never for the Business, Economic and Social (which includes all Humanities) sciences.

Policy inferences are immediate.

The productivity impact of selective funding allocation is enhanced if it is more responsive to prior research performance. Funding needs to go to the strongest researchers.

But the evidence suggest that the marginal returns from raised funding are savagely diminishing. In the South African case, even for the most productive research chair recipients, the cost per additional publication is 22 times as high as comparable researchers without funding, and 32 times as high per citation. If the objective of research funding is to raise the level of output and impact across and entire research system, a more broad-based inclusive funding approach that gives smaller awards to more researchers may carry more promise.

Differential rates of return across academic disciplines also suggests that funding allocations that adjusts funding allocations to reflect these differences can raise aggregate levels of output and impact.

Finally, if funding allocation is to follow revealed productivity, productivity has to be transparently and objectively monitored. Use of the growing number of objective bibliometric measures at least in part in reaching allocative decisions, is an obvious supplement to the reliance on peer review alone. All the more since peer review is itself not immune from bias – as the South African case demonstrates.

Research Brief 20
1 April 2015

Related South Africa’s Cities and Growth Spatial Challenges and Policy Interventions Content

Request for Proposals: The role of cities as drivers of growth and employment
Background Urbanization in South Africa is expected to reach 80% by...
Call for Work
South Africa’s future will be decided in our cities
Discussion Document 14 South Africa’s cities face multiple, overlap...
Dieter von Fintel, Justin Visagie, Ivan Turok, Takwanisa Machemedze, Claus Rabe, Sebastian Galiani, Edward Glaeser
Discussion Document
Monitoring South Africa’s metropolitan economies: A survey of the data landscape
Discussion Document 13 Disparities in data across different metropo...
Dieter von Fintel
Discussion Document
Cities, productivity and Jobs in SA: Problems and potential
Discussion Document 12 Cities contribute to national prosperity bec...
Ivan Turok, Justin Visagie
Discussion Document
Place-based economic policies: international lessons for South Africa
Discussion Document 11 Place-based policies are designed to support...
Harris Selod, Claus Rabe
Discussion Document
What luminosity data can and cannot reveal about South Africa’s urban economies
Discussion Document 10 As novel types of data are becoming availabl...
Takwanisa Machemedze
Discussion Document
Crime: A policy-oriented survey
Discussion Document 9 South Africa has a reputation for having high...
Sebastian Galiani
Discussion Document
Virtual CDE Workshop on SA Cities and Growth
Urban economics has provided powerful insights into how the charact...

Search Resources

Ground Floor Brookside Building
11 Imam Haron Road
Claremont, 7700
Cape Town

PostNet Suite # 109
Private Bag X1005
Claremont 7735
Cape Town

Get Social