Assessing the sensitivity of global sustainability indicators to different composite index methods: application to cereal farming in Catalonia
Abstract
Aim of study: Forward-thinking farming must become more efficient and sustainable to address the multitude of challenges it faces. This study examines the sensitivity of sustainability assessments to different composite index construction methodologies. Area of study: We compare five distinct approaches to composite sustainability index (CSI) construction using Farm Accountancy Data Network (FADN)-EU data from 2012 to 2022. Environmental, economic, and social dimensions are incorporated to provide comprehensive sustainability assessments for cereal farming in Catalonia, Spain. Material and methods: Through random effects generalized least squares regression and variance-based sensitivity analysis, we demonstrated that methodological choices significantly influence sustainability outcomes, with some methods showing greater robustness and theoretical consistency than others. Our analysis operated within the weak sustainability paradigm using additive aggregation methods. Main results: Our findings revealed that expert weighting produced results remarkably similar to equal weighting in this context, while factor analysis with min-max normalization offered the most theoretically consistent results. Principal component analysis provided strong statistical stability but had limitations for quantitative data applications. Research highlights: This study provides the first systematic integration of variance-based sensitivity analysis with econometric validation for agricultural sustainability indices, developing a quantitative robustness scoring framework that enables evidence-based method selection. The methodological insights contribute to enhanced transparency and reliability in agricultural sustainability assessment within the weak sustainability framework.
Downloads
References
Arendonk AV, 2015. The development of the share of agriculture in GDP and employment: A case study of China, Indonesia, the Netherlands and the United States. Wageningen University, Leibniz Institute of Agricultural Development in Transition Economies.
Brennan M, Hennessy T, Dillon E, Meredith D, 2023. Putting social into agricultural sustainability: Integrating assessments of quality of life and wellbeing into farm sustainability indicators. Sociol. Rural, 63(3), pp.629-660. https://doi.org/10.1111/soru.12417
Boas I, Biermann F, Kanie N, 2016. Cross-sectoral strategies in global sustainability governance: Towards a nexus approach. Int. Environ. Agreem.-Polit. Law Econom, 16(3), 449–464. https://doi.org/10.1007/s10784-016-9321-1
Böhringer C, Jochem PE, 2007. Measuring the immeasurable—A survey of sustainability indices. Ecol. Econ, 63(1), 1-8. https://doi.org/10.1016/j.ecolecon.2007.03.008
Caccavale OM, Giuffrida V, 2020. The Proteus composite index: Towards a better metric for global food security. World Dev, 126, 104709. https://doi.org/10.1016/j.worlddev.2019.104709
Cardillo C, Di Fonzo A, Liberati C. (2023). The farm’s orientation towards sustainability: An assessment using FADN data in Italy. Land, 12(2), 301. https://doi.org/10.3390/land12020301
Chatterjee S, Hadi AS, 2009. Sensitivity analysis in linear regression. John Wiley & Sons.
Chopin P, Mubaya CP, Descheemaeker K, Öborn I, Bergkvist G, 2021. Avenues for improving farming sustainability assessment with upgraded tools, sustainability framing and indicators. A review. AGRON SUSTAIN DEV, 41(2), p.19. https://doi.org/10.1007/s13593-021-00674-3
Colnago P, Dogliotti S, 2020. Introducing labour productivity analysis in a co-innovation process to improve sustainability in mixed family farming. Agric. Syst, 177, p.102732. https://doi.org/10.1016/j.agsy.2019.102732
Darnhofer I, Fairweather J, Moller H, 2010. Assessing a farm’s sustainability: Insights from resilience thinking. Int.J. Agric. Sustain, 8(3), 186–198. https://doi.org/10.3763/ijas.2010.0480
Dabkiene V, Balezentis T, Streimikiene D, 2021. Development of agri-environmental footprint indicator using the FADN data: Tracking development of sustainable agricultural development in Eastern Europe. Sustain. Prod. Consum, 27, 2121-2133. https://doi.org/10.1016/j.spc.2021.05.017
Deytieux V, Munier-Jolain N, Caneill, J, 2016. Assessing the sustainability of cropping systems in single- and multi-site studies. A review of methods. Eur. J. Agron, 72, 107–126. https://doi.org/10.1016/j.eja.2015.10.005
Doré T, Makowski D, Malézieux E, Munier-Jolain NG, Tchamitchian M, Tittonell P, 2011. Facing up to the paradigm of ecological intensification in agronomy: Revisiting methods, concepts and knowledge. Eur. J. Agron, 34(4), 197–210. https://doi.org/10.1016/j.eja.2011.02.006
Ehrmann M, 2008. Comparing sustainable value approach, data envelopment analysis and indicator approaches-An application on German dairy farms. https://doi.org/10.22004/AG.ECON.44140
Ehrmann M, 2010. Assessing ecological and economic impacts of policy scenarios on farm level: Vortrag anlässlich der 50. Jahrestagung der GEWISOLA „Möglichkeiten und Grenzen der wissenschaftlichen Politikanalyse “Braunschweig, 29.09.–01.10. 2010. https://doi.org/10.22004/AG.ECON.93949
FAO, 2014. Sustainability Assessment of Food and Agriculture Systems: SAFA, Guidelines, Version 3.0. Food and Agriculture Organization of the United Nations.
Fisher E, Hellin J, Greatrex H, Jensen N, 2019. Index insurance and climate risk management: Addressing social equity. Dev. Policy Rev, 37(5), 581-602. https://doi.org/10.1111/dpr.12387
Gan X, Fernández IC, Guo J, Wilson M, Zhao Y, Zhou B, Wu J, 2017. When to use what: Methods for weighting and aggregating sustainability indicators. Ecol. Indic, 81, 491-502. https://doi.org/10.1016/j.ecolind.2017.05.068
Gómez-Limón JA, Sanchez-Fernandez G, 2010. Empirical evaluation of agricultural sustainability using composite indicators. Ecol. Econ, 69(5), 1062-1075. https://doi.org/10.1016/j.ecolecon.2009.11.027
Gómez-Limón JA, Arriaza M, Guerrero-Baena MD, 2020. Building a composite indicator to measure environmental sustainability using alternative weighting methods. Sustainability, 12(11), 4398. https://doi.org/10.3390/su12114398
González-García S, Rama M, Cortés A, García-Guaita F, Núñez A, Louro LG, Moreira MT, Feijoo G, 2019. Embedding environmental, economic and social indicators in the evaluation of the sustainability of the municipalities of Galicia (northwest of Spain). J. Clean. Prod, 234, 27-42. https://doi.org/10.1016/j.jclepro.2019.06.158
Grenz J, Thalmann C, Stämpfli A, Studer C, Häni F, 2009. RISE–a method for assessing the sustainability of agricultural production at farm level. Rural Development News, 1(2009), pp.5-9.
Halbe J, Adamowski J, 2019. Modeling sustainability visions: Acasestudy ofmulti-scale food systemsin southwestern Ontario. J. Environ. Manage, 231, 1028–1047. https://doi.org/10.1016/j.jenvman.2018.09.099
Jane Dillon E, Hennessy T, Buckley C, Donnellan T, Hanrahan K, Moran B, Ryan M, 2016. Measuring progress in agricultural sustainability to support policy-making. International Int. J. Agric. Sustain, 14(1), pp.31-44. https://doi.org/10.1080/14735903.2015.1012413
Jain N, Mohapatra G, 2023. A comparative assessment of Composite Environmental Sustainability Index for emerging economies: a multidimensional approach. Manag. Environ. Qual. Int. J. Vol. 34 No. 5. https://doi.org/10.1108/meq-12-2022-0330
Karimian Abdar Z, Amirtaimoori S, Zare Mehrjerdi MR, Mehrabi Boshrabadi H, 2022. A composite index for assessment of agricultural sustainability: the case of Iran. Environ. Sci. Pollut. Res. 29:47337–47349. https://doi.org/10.1007/s11356-022-19154-6
Kelly E, Ryan M, Finn J, Hennessy T, 2015. Farm-level indicators for evaluating sustainability and emerging new policy topics. Flint Project.
Kelly E, Latruffe L, Desjeux Y, Ryan M, Uthes S, Diazabakana A, Finn J, 2018. Sustainability indicators for improvedassessment of the effects of agricultural policy across the EU: Is FADN the answer? Ecol. Indic, 89, 903-911. https://doi.org/10.1016/j.ecolind.2017.12.053
Kristjanpoller W, Minutolo MC, 2018. A hybrid volatility forecasting framework integrating GARCH, artificial neural network, technical analysis and principal components analysis. Expert Syst. Appl, 109, 1–11. https://doi.org/10.1016/j.eswa.2018.05.011
Latruffe L, Diazabakana A, Bockstaller C, Desjeux Y, Finn J, Kelly E, Ryan M, Uthes S, 2016. Measurement of sustainability in agriculture: a review of indicators. Stud. Agr. Econ, 118(3), pp.123-130. https://doi.org/10.7896/j.1624
Malak-Rawlikowska A, Majewski E, Was A, Borgen SO, Csillag P, Donati M, Freeman R, Hoàng V, Lecoeur JL, Mancini MC, Nguyen A, Saïdi M, Tocco B, Török Á, Veneziani M, Vittersø G, Wavresky P, 2019. Measuring the economic, environmental, and social sustainability of short food supply chains. Sustainability, 11(15), 4004. https://doi.org/10.3390/su11154004
McNeill D, 2019. The contested discourse of sustainable agriculture, Glob. Policy, 10(S1), 16–27. https://doi.org/10.1111/1758-5899.12603
Mishra B, Gyawali BR, Paudel KP, Poudyal NC, Simon MF, Dasgupta S, Antonious G, 2018. Adoption of sustainable agriculture practices among farmers in Kentucky, USA. J. Environ. Manag, 62(6), pp.1060-1072. https://doi.org/10.1007/s00267-018-1109-3
Moreno-Miranda C, Moreno R, Moreno P, 2020. Protected-Denomination-of-Origin cocoa bean: Chain governance and sustainability performance. Econom. agro-aliment, 22(1), 1–24. https://doi.org/10.3280/ecag1-2020oa10066
Nardo M, Saisana M, Saltelli A, Tarantola S, 2005. Tools for composite indicators building. Joint Research Centre. European Commission. EUR 21682 EN.
Nchanji EB, Bellwood-Howard I, Schareika N, Chagomoka T, Schlesinger J, Axel D, Rüdiger G, 2017. Assessing the sustainability of vegetable production practices in northern ghana. Int. J. Agric. Sustain, 15(3), 321–337. https://doi.org/10.1080/14735903.2017.1312796
OECD, 2008: Handbook on Constructing Composite Indicators: Methodology and User Guide. OECD publishing.
Pandey D, 2020. Agricultural sustainability and climate change nexus. Contemporary environmental issues and challenges in era of climate change, 77-97. https://doi.org/10.1007/978-981-32-9595-7_4
Pollesch N, Dale V, 2015. Applications of aggregation theory to sustainability assessment. Ecol. Econ. 114, 117–127. https://doi.org/10.1016/j.ecolecon.2015.03.011
Pollesch NL, Dale VH, 2016. Normalization in sustainability assessment: Methods and implications. Ecol. Econ, 130, 195-208. https://doi.org/10.1016/j.ecolecon.2016.06.018
Ryan M, Hennessy T, Buckley C, Dillon EJ, Donnellan T, Hanrahan K, Moran B, 2016. Developing farm-level sustainability indicators for Ireland using the Teagasc National Farm Survey. Ir. j. agric. food res, 55(2), 112-125. https://doi.org/10.1515/ijafr-2016-0011
Sachs JD, Schmidt-Traub G, Kroll C, Lafortune G, Fuller G, 2019. Sustainable Development Report 2019: Transformations to Achieve the Sustainable Development Goals. Bertelsmann Stiftung and Sustainable Development Solutions Network. https://doi.org/10.18356/af2e028c-en
Sahoo G, Wani AM, Satpathy B, Rout S, 2020. Agriculture and Rural Development for Sustainability. J. Agric. Rural Dev. Stud: Spatial Issues, Challenges and Approaches, 75. https://doi.org/10.22161/conf.aerd.2020
Sala S, Cerutti AK, Pant R, 2018. Development of a weighting approach for the Environmental Footprint. Publications Office of the European Union, Luxembourg. ISBN 978-92-79-68042-7.
Sulewski P, Kłoczko-Gajewska A, 2018. Development of the sustainability index of farms based on surveys and FADN sample. Probl. Agric. Econ, 3(356). https://doi.org/10.30858/zer/94474
Talukder B, W. Hipel K, W. vanLoon G, 2017. Developing composite indicators for agricultural sustainability assessment: Effect of normalization and aggregation techniques. Resources, 6(4), p.66. https://doi.org/10.3390/resources6040066
Thies C, Kieckhäfer K, Spengler TS, Sodhi MS, 2019. Operations research for sustainability assessment of products: A review. Eur. J. Oper. Res, 274(1), 1–21. https://doi.org/10.1016/j.ejor.2018.04.039
Tiwari K, Khan MS, 2019. An action research approach for measurement of sustainability in a multi-echelon supply chain: Evidences from Indian sea food supply chains. J. Clean. Prod, 235, 225–244. https://doi.org/10.1016/j.jclepro.2019.06.200
Usubiaga-Liaño A, Ekins P, 2024. Methodological choices for reflecting strong sustainability in composite indices. Ecol. Econ, 221, 108192. https://doi.org/10.1016/j.ecolecon.2024.108192
United Nations, 2015. Global Sustainable Development Report, GSDR. UNDESA. https://doi.org/10.18356/d4d22c19-en
Van Cauwenbergh N, Biala K, Bielders C, Brouckaert V, Franchois L, Cidad VG, Hermy M, Mathijs E, Muys B, Reijnders J, Sauvenier X, 2007. SAFE—A hierarchical framework for assessing the sustainability of agricultural systems. Agric. Ecosyst. Environ, 120(2-4), pp.229-242. https://doi.org/10.1016/j.agee.2006.09.006
World Bank, 2013. The world bank annual report 2013. The World Bank. https://doi.org/10.1596/978-0-8213-9937-8
Wu J, Wu T, 2012. Sustainability indicators and indices: an overview. Handbook of Sustainable Management. Imperial College Press, London, pp. 65–86. https://doi.org/10.1142/9789814354820_0004
Yong AG, Pearce S, 2013. A beginner’s guide to factor analysis: Focusing on exploratory factor analysis. Tutor. quant. methods psychol, 9(2), 79-94. https://doi.org/10.20982/tqmp.09.2.p079
Copyright (c) 2026 Consejo Superior de Investigaciones Científicas (CSIC)
This work is licensed under a Creative Commons Attribution 4.0 International License.
© CSIC. Manuscripts published in both the print and online versions of this journal are the property of the Consejo Superior de Investigaciones Científicas, and quoting this source is a requirement for any partial or full reproduction.
All contents of this electronic edition, except where otherwise noted, are distributed under a Creative Commons Attribution 4.0 International (CC BY 4.0) licence. You may read the basic information and the legal text of the licence. The indication of the CC BY 4.0 licence must be expressly stated in this way when necessary.
Self-archiving in repositories, personal webpages or similar, of any version other than the final version of the work produced by the publisher, is not allowed.
