Publications
ERF-AISBL often produce publications and position papers. Details can be found below.
Key performance indicators of Research Infrastructures / 2
Following the recent Competitiveness Council Conclusions, which mandate ESFRI to develop a common approach for monitoring the performance of RIs, a survey was conducted by ERF-AISBL, in cooperation with CERIC-ERIC, to the community of European RIs to get their insight into how they address (or would address) the issue of Key Performance Indicators. See here the results of this survey. The Executive Summary and Conclusion are also available below.
Executive summary
Following the recent Competitiveness Council Conclusions, which mandate ESFRI to develop a common approach for monitoring the performance of research infrastructures (RIs), a questionnaire was sent by ERF-AISBL (Association of European-level Research Infrastructure Facilities) to the community of European RIs in order to gain a better insight into how they address (or would address as the case may be) the issue of Key Performance Indicators (KPIs). 36 replies were received, as listed in Annex 1. The results indicate a rather strong coherence with the proposed principles across the landscape of RIs.
While half of the respondents already have KPIs in place, the other half agree that they should have them. Respondents also believe that KPIs should be used in the strategic management of the institutions and, as such, adopted by, and reported to the decision-making bodies of the RIs. There is also a strong preference for their publication, although some RIs stress the importance of putting KPIs into context when making them available publicly to ensure clarity, but also because, without such contextual information, the performance cannot be reliably compared across RIs.
The quality of indicators is considered to be highly important by the RIs. Respondents believe indeed that indicators should be “relevant, accepted, credible, easy to monitor and robust”, although only three respondents report that their current KPIs already meet these criteria. The prevailing opinion of the RIs is that KPIs should be linked to the objectives of their institutions. Considering that RIs pursue some objectives that are specific to each of them, a number of respondents warn against prescribing the use of the same indicators by all RIs in a ‘top-down’ approach. Additionally, some respondents emphasize that, in addition to quantitative KPIs, attention should also be devoted to non-measurable, qualitative performance criteria and propose that KPIs be accompanied by case studies and other narratives in order to appropriately present progress in the pursuit of the objectives of their infrastructure. You can read the full report here.
While half of the respondents already have KPIs in place, the other half agree that they should have them. Respondents also believe that KPIs should be used in the strategic management of the institutions and, as such, adopted by, and reported to the decision-making bodies of the RIs. There is also a strong preference for their publication, although some RIs stress the importance of putting KPIs into context when making them available publicly to ensure clarity, but also because, without such contextual information, the performance cannot be reliably compared across RIs.
The quality of indicators is considered to be highly important by the RIs. Respondents believe indeed that indicators should be “relevant, accepted, credible, easy to monitor and robust”, although only three respondents report that their current KPIs already meet these criteria. The prevailing opinion of the RIs is that KPIs should be linked to the objectives of their institutions. Considering that RIs pursue some objectives that are specific to each of them, a number of respondents warn against prescribing the use of the same indicators by all RIs in a ‘top-down’ approach. Additionally, some respondents emphasize that, in addition to quantitative KPIs, attention should also be devoted to non-measurable, qualitative performance criteria and propose that KPIs be accompanied by case studies and other narratives in order to appropriately present progress in the pursuit of the objectives of their infrastructure. You can read the full report here.
Conclusion
While the recent Council Conclusions underlined the importance of performance management, care needs to be given in designing and implementing it correctly. One should consider the principles behind performance management and respect the balance between its objectives and the administrative burden it can impose on the staff. Implemented in such a balanced way, KPIs can significantly contribute to the long-term sustainability of a research infrastructure, as funders base their investment decisions also on the results and impacts of the initiatives they fund, and such data can help RI managers deliver what is expected from them. Furthermore, providing quality data on the performance of RIs not only contributes to the notion that they have a credible, output-oriented management in place, but also provides assurance that the various objectives that supported their establishment are indeed effectively pursued.
This full report was prepared by Jana Kolar (CERIC-ERIC), Andrew Harrison (Diamond Light Source Ltd) and Florian Gliksohn (ELI-DC) for ERF. CERIC-ERIC was supported by the ACCELERATE project, funded by the European Union Framework Programme for Research and Innovation Horizon 2020, under grant agreement 731112. The authors would like to sincerely thank the participating research infrastructures for their invaluable input.
This full report was prepared by Jana Kolar (CERIC-ERIC), Andrew Harrison (Diamond Light Source Ltd) and Florian Gliksohn (ELI-DC) for ERF. CERIC-ERIC was supported by the ACCELERATE project, funded by the European Union Framework Programme for Research and Innovation Horizon 2020, under grant agreement 731112. The authors would like to sincerely thank the participating research infrastructures for their invaluable input.