Research Performance Evaluation a systems approach

The "bang for buck" is an important question directed at the biomedical research community especially in light of the disproportionate results against the billions of dollars invested in research activities.

For a funding agency answering the question is not only important to justify their past decisions but also to make well informed decisions while investing limited funds.

Precise and justified evaluation can also vindicate the activities of an entire research community.

A careful evaluation of the current outcome measures and indices reveals their shallow and short term outlook in analyzing the performance of research institutes and researchers.

In this presentation first, we explore the problem of how to create a streamlined system that allows for a scholar and fair peer-review system for allocation of research funds in biomedical research. A special focus is given to the problem associated with the scarcity of expert peer-reviewers willing to serve on the very time-demanding task of reviewing proposals. Second, we explore the problem of how to create a research performance evaluation system that can be used for post-award performance evaluation. Complexities inherent to this problem are explored, including the different perspectives that can be taken on the evaluation process (academic, economic, and healthcare), the system non-linearities (e.g., concentrating on publications will reduce available time for patents), and the field-specificity of these metrics (e.g., a researcher in cardiology is fairly different from a researcher in biochemistry). Last, we explore the problem of evaluating the healthcare return on investment in clinical research. Special attention is paid to the delays and spillover effects in this system, common to most projects focusing on long-term infrastructure and that frequently confuse its evaluation plan. While covering each of the the three topics, we make use of a System Dynamics methodology, with simple models that will walk readers through some of the intricacies of the problem, potential policy solutions, and future areas that deserve deeper analysis.