Understanding and evaluating the impact of integrated problem-oriented research programmes: Concepts and considerations
Researchers and research organizations are under increasing pressure to demonstrate that their work contributes to positive change and helps solve pressing societal challenges. There is a simultaneous trend towards more engaged transdisciplinary research that is complexity-aware and appreciates that...
| Main Authors: | , |
|---|---|
| Format: | Journal Article |
| Language: | Inglés |
| Published: |
Oxford University Press
2021
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10568/111879 |
| _version_ | 1855521578041737216 |
|---|---|
| author | Belcher, B. Hughes, K. |
| author_browse | Belcher, B. Hughes, K. |
| author_facet | Belcher, B. Hughes, K. |
| author_sort | Belcher, B. |
| collection | Repository of Agricultural Research Outputs (CGSpace) |
| description | Researchers and research organizations are under increasing pressure to demonstrate that their work contributes to positive change and helps solve pressing societal challenges. There is a simultaneous trend towards more engaged transdisciplinary research that is complexity-aware and appreciates that change happens through systems transformation, not only through technological innovation. Appropriate evaluation approaches are needed to evidence research impact and generate learning for continual improvement. This is challenging in any research field, but especially for research that crosses disciplinary boundaries and intervenes in complex systems. Moreover, evaluation challenges at the project scale are compounded at the programme scale. The Forest, Trees and Agroforestry (FTA) research programme serves as an example of this evolution in research approach and the resulting evaluation challenges. FTA research is responding to the demand for greater impact with more engaged research following multiple pathways. However, research impact assessment in the CGIAR (Consultative Group on International Agricultural Research) was developed in a technology-centric context where counterfactual approaches of causal inference (experimental and quasi-experimental) predominate. Relying solely on such approaches is inappropriate for evaluating research contributions that target policy and institutional change and systems transformation. Instead, we propose a multifaceted, multi-scale, theory-based evaluation approach. This includes nested project- and programme-scale theories of change (ToCs); research quality assessment; theory-based outcome evaluations to empirically test ToCs and assess policy, institutional, and practice influence; experimental and quasi-experimental impact of FTA-informed ‘large n’ innovations; ex ante impact assessment to estimate potential impacts at scale; and logically and plausibly linking programme-level outcomes to secondary data on development and conservation status. |
| format | Journal Article |
| id | CGSpace111879 |
| institution | CGIAR Consortium |
| language | Inglés |
| publishDate | 2021 |
| publishDateRange | 2021 |
| publishDateSort | 2021 |
| publisher | Oxford University Press |
| publisherStr | Oxford University Press |
| record_format | dspace |
| spelling | CGSpace1118792024-08-27T10:35:34Z Understanding and evaluating the impact of integrated problem-oriented research programmes: Concepts and considerations Belcher, B. Hughes, K. impact assessment evaluation research innovation development policies Researchers and research organizations are under increasing pressure to demonstrate that their work contributes to positive change and helps solve pressing societal challenges. There is a simultaneous trend towards more engaged transdisciplinary research that is complexity-aware and appreciates that change happens through systems transformation, not only through technological innovation. Appropriate evaluation approaches are needed to evidence research impact and generate learning for continual improvement. This is challenging in any research field, but especially for research that crosses disciplinary boundaries and intervenes in complex systems. Moreover, evaluation challenges at the project scale are compounded at the programme scale. The Forest, Trees and Agroforestry (FTA) research programme serves as an example of this evolution in research approach and the resulting evaluation challenges. FTA research is responding to the demand for greater impact with more engaged research following multiple pathways. However, research impact assessment in the CGIAR (Consultative Group on International Agricultural Research) was developed in a technology-centric context where counterfactual approaches of causal inference (experimental and quasi-experimental) predominate. Relying solely on such approaches is inappropriate for evaluating research contributions that target policy and institutional change and systems transformation. Instead, we propose a multifaceted, multi-scale, theory-based evaluation approach. This includes nested project- and programme-scale theories of change (ToCs); research quality assessment; theory-based outcome evaluations to empirically test ToCs and assess policy, institutional, and practice influence; experimental and quasi-experimental impact of FTA-informed ‘large n’ innovations; ex ante impact assessment to estimate potential impacts at scale; and logically and plausibly linking programme-level outcomes to secondary data on development and conservation status. 2021-10-19 2021-03-08T08:14:49Z 2021-03-08T08:14:49Z Journal Article https://hdl.handle.net/10568/111879 en Open Access Oxford University Press Belcher, B.M. and Hughes, K., 2020. Understanding and evaluating the impact of integrated problem-oriented research programmes: Concepts and considerations. Research Evaluation. https://doi.org/10.1093/reseval/rvaa024 |
| spellingShingle | impact assessment evaluation research innovation development policies Belcher, B. Hughes, K. Understanding and evaluating the impact of integrated problem-oriented research programmes: Concepts and considerations |
| title | Understanding and evaluating the impact of integrated problem-oriented research programmes: Concepts and considerations |
| title_full | Understanding and evaluating the impact of integrated problem-oriented research programmes: Concepts and considerations |
| title_fullStr | Understanding and evaluating the impact of integrated problem-oriented research programmes: Concepts and considerations |
| title_full_unstemmed | Understanding and evaluating the impact of integrated problem-oriented research programmes: Concepts and considerations |
| title_short | Understanding and evaluating the impact of integrated problem-oriented research programmes: Concepts and considerations |
| title_sort | understanding and evaluating the impact of integrated problem oriented research programmes concepts and considerations |
| topic | impact assessment evaluation research innovation development policies |
| url | https://hdl.handle.net/10568/111879 |
| work_keys_str_mv | AT belcherb understandingandevaluatingtheimpactofintegratedproblemorientedresearchprogrammesconceptsandconsiderations AT hughesk understandingandevaluatingtheimpactofintegratedproblemorientedresearchprogrammesconceptsandconsiderations |