Journal Information
Vol. 43. Issue. 6.November - December 2023
Pages 663-806
Vol. 43. Issue. 6.November - December 2023
Pages 663-806
Original article
Full text access
Value of concerted and hospital hemodialysis through a multi-criteria decision analysis
Valor de la hemodiálisis concertada y la hospitalaria mediante un análisis de decisión multicriterio
Araceli Caro Martíneza,b, María del Carmen Valcárcel Cabreraa, Antonio Olry de Labry Limac,d,e,
Corresponding author
a Escuela Andaluza de Salud Pública, Granada, Spain
b Programa de Doctorado Interuniversitario en Ciencias de la Salud, Universidad de Jaén-Universidad de Sevilla (UJA-US), Sevilla, Spain
c Centro Andaluz de Documentación e Información de Medicamentos (CADIME), Escuela Andaluza de Salud Pública, Granada, Spain
d Consorcio de Investigación Biomédica en Red Epidemiología y Salud Pública, CIBERESP
e Instituto de Investigación Biosanitaria ibs.GRANADA, Granada, Spain
This item has received
Article information
Full Text
Download PDF
Figures (4)
Show moreShow less
Additional material (1)

To evaluate the value of the provision of contracted versus hospital dialysis services for the treatment of chronic kidney disease in Spain using the multicriteria decision analysis methodology.


The EVIDEM (Evidence and Value: Impact on Decision Making) evaluation framework was used to calculate the estimated value of both dialysis delivery models (arranged vs. hospital) through a virtual workshop in which different profiles participated: directors and managers, professionals and heads of units and representatives of patients and relatives. The scores were combined using an additive lineal model, which combined the weight of the model with the individual score of the criteria, and each value was transformed to a scale between 0 and 1.


The estimated value for arranged dialysis was 0.29 (DS: ±0.2) and 0.39 (DS: ±0.2) for hospital dialysis. All profiles gave a higher value to hospital hemodialysis compared to contracted hemodialysis. The highest value for hospital dialysis was for patients (0.44), with the lowest mean value for directors (0.36) and the range for arranged dialysis being between patients (0.31) and intermediate positions (0.27).


Hospital hemodialysis obtained a higher value than concerted dialysis. In general, the panelists affirmed that it is a useful and interesting exercise and that, to a certain extent, it provides security in decision-making, since it allows ordering, rationalizing and considering, in an explicit and transparent manner, the different criteria involved.

Decision making
Health technology assessment
External services

Evaluar el valor de la prestación de servicios de diálisis concertada frente a la hospitalaria para el tratamiento de la enfermedad renal crónica en España mediante la metodología de análisis de decisión multicriterio.


Se utilizó el marco de evaluación EVIDEM (Evidence and Value: Impact on Decision Making) para el cálculo del valor estimado de ambos modelos de prestación de la diálisis (concertada vs. hospitalaria) mediante un taller virtual en el que participaron diferentes perfiles: directivos y gestores, profesionales y responsables de unidades y representantes de pacientes y familiares. Las puntuaciones se combinaron mediante un modelo lineal aditivo, que combinó la ponderación del modelo con la puntuación individual de los criterios, y cada valor se transformó a una escala entre el 0 y 1.


La estimación del valor para la diálisis concertada fue de 0,29 (DE: ±0,2) y de 0,39 (DE: ±0,2) para la diálisis hospitalaria. Todos los perfiles otorgaron un mayor valor a la hemodiálisis hospitalaria. El mayor valor para la diálisis hospitalaria fue de los pacientes (0,44); el menor valor medio fue de los directivos (0,36) y el rango para la diálisis concertada estuvo entre los pacientes (0,31) y los cargos intermedios (0,27).


La hemodiálisis hospitalaria obtuvo un mayor valor que la diálisis concertada. En general, los panelistas afirmaron que resulta un ejercicio útil e interesante y que, en cierta medida, aporta seguridad en la toma de decisiones, ya que permite ordenar, racionalizar y considerar, de manera explícita y transparente, los diferentes criterios involucrados.

Palabras clave:
Toma de decisiones
Evaluación de tecnologías sanitarias
Servicios externos
Full Text

Decision-makers and healthcare managers must introduce novel healthcare technologies without losing sight of the economic sustainability of the system.1 Different tools and methodologies for planning, prioritization, and allocation of resources are available for decision-making, including budget impact and economic evaluation.2 However, these tools do not consider criteria such as equity or burden of disease, or at least not explicitly.3–5

In the field of health technology assessment, multicriteria decision analysis (MCDA) is emerging as a promising tool.3 MCDA is a set of methods that, complementing other techniques, aids decision-making in evidence-based prioritization and use of resources and enables practitioners to address complex problems across multiple dimensions.5 The MCDA is a methodology that explicitly orders and assists in deliberative processes, i.e., it provides support without prescriptive value. Thus, it incorporates different values and interests in an explicit and transparent manner, leading to improved engagement, transparency, and accountability.6–8 The growing literature on MCDA in healthcare decisions points to three reasons why this group of methods may represent an alternative to economic evaluation in STD procedures.9,10 First, the explicit inclusion of a list of value dimensions; second, the assignment of quantitative weights among the different evaluation criteria, so that their relative importance is also explicit, which improves the transparency of the preference bidding process and the participation of the agents involved; and third, the possibility of including all of them in the value assessment, which contributes to the legitimacy of the process.9,10

The aim of this study was to evaluate the value of outsourced versus in-hospital dialysis service provision for the treatment of chronic kidney disease (CKD) in Spain using the MCDA methodology. The advantage of comparing these two types of service provision with MCDA is that it allows us to incorporate multiple elements, beyond efficacy, safety, and price in making decisions on financing, provision, and prioritization of health resources, when presenting two alternatives in the provision of a care service.


In a multi-criteria decision context, the main issues are: 1) problem structuring and identification of alternatives; 2) evidence review; 3) modeling (identifying and establishing evaluation criteria for the different alternatives; measuring and prioritizing interventions according to the established criteria; weighting to obtain the relative importance of each criterion and an overall evaluation of each intervention) and 4) deliberation.3

The development of the MCDA workshop was carried out virtually and used the EVIDEM (Evidence and Value: Impact on DEcision-Making) framework, which is freely available through its website (The EVIDEM Collaboration).11 This framework consists of 22 criteria (15 quantitative and seven qualitative) and requires that evidence meet three characteristics: 1) broad (covering all areas related to the health issue in question); 2) relevant (key to the decision) and 3) explicit, as a key element of transparency and accountability in decision-making.

To carry out the MCDA, a group of 23 participants was formed, balancing different levels of responsibility, from the Andalusian Public Health System (SSPA) made up of directors and managers (n = 3), representatives of the Association for the Fight Against Kidney Diseases (ALCER) (n = 2), health evaluation and management professionals (n = 5), healthcare practitioners (n = 3), nephrology nursing professionals (n = 5), and nephrology unit managers (n = 3). In addition, to include other organizational perspectives, two supervising nephrologists from other autonomous communities were invited.

In the first part of the workshop, participants were offered a training session on the MCDA methodology as a decision-making tool by an expert in health technology assessment.

In the second part, and before the participants were familiarized with the methodology under study, weights were assigned to the evaluation framework: each participant scored each criterion according to importance between 1 (least relevant) and 5 (most relevant).

Subsequently, participants were presented with the results of the scoping review of the literature on the subject, according to the different domains of the evaluation framework.12 For this review, the following databases were consulted: PubMed, Embase, Web of Science, CINAHL, Scopus, and The Cochrane Library. We included articles comparing outsourced versus hospital dialysis, according to the EVIDEM framework criteria.

After this, each participant scored the absolute criteria (those not involving a comparison between alternatives) between 0 (lowest value) and 5 (highest value) and the relative criteria (involving a comparison between both dialysis delivery modalities) between −5 and 5 points. Finally, an evaluation was made of the 7 qualitative criteria, based on the available evidence and the experience of the participants.

The estimated value for each intervention was obtained using a linear additive model, combining the relative weighting of each criterion with the score reported for each intervention. Each estimated value was transformed into a scale from 0 to 100, to facilitate its interpretation and its standard deviation (SD), which represents the synthesis of the overall value of each alternative as an expression of the degree to which one of them is preferred to another. To check the degree of consistency and replicability of the analysis, the model was weighted by scoring the criteria from 1 to 5.

Finally, although different programs are available for multicriteria decision analysis,13 we designed a tool in Microsoft Excel format that allowed the information collected to be obtained and displayed in a more explicit, dynamic, and transparent way. At the end of the workshop, the participants discussed the results, and held a final debate on the contribution of this methodology and its application to decision-making in different areas of nephrology.


In the first part of the model weighting, workshop participants scored all criteria above 3 (out of 5), except the indirect cost criterion, which scored 2.8 (SD: ±0.8). The highest mean score was 4.3 for the criteria “comparative effectiveness” (SD: ±0.8) and “comparative safety/tolerability” (SD: ±1.0) (Fig. 1). We observed high consistency between the scores using the 0–5 scale and the 0–100 scale (Figs. S1A–S1F in Appendix A show the mean score according to professional profile).

Fig. 1.

Average weight of the EVIDEM value assessment on a scale of 1–5.


With regard to the absolute criteria scores, in the hospital dialysis option, the criterion with the highest mean score was “unmet needs” with 3.8 (out of 5) and high consistency (SD: ±0.8); the other criteria with the highest mean scores were “severity of disease,” “quality of evidence,” and “type of therapeutic benefit” with a score of 3.3, with a standard deviation ranging from 0.8 to 1.2. In contrast, the criterion with the lowest score was “type of preventive benefit” with score 2.3 (SD: ±1.0). As for the comparative criteria, the criterion with the highest score was “comparative safety/tolerability” with a mean score of 1.2 (SD: ±2.6); the remaining criteria ranged from 0.3 (SD: ±3) for “patient-reported outcomes” to −0.2 (SD: ±3.1) for “indirect costs” and “fixed health care costs” (SD: ±2.5).

In the outsourced dialysis option, the highest mean score was 3.7 (SD: ±1.2) for the criterion “size of the affected population,” followed by “unmet need” with a score of 3.5 (SD: ±0.5). In contrast, the lowest mean score was for "type of preventive benefit” with a mean of 2.3 (SD: ±1.5). The relative criteria scores ranged from 0.8 (SD: ±3.3) for the criterion “comparative costs of the intervention.” The lowest were for “compared efficacy/effectiveness” and “compared non-health costs,” with a score of −0.8.

When the scores were analyzed according to the profile of the participants, it was observed that, in general, executives and managers gave a higher score to those dimensions related to cost, whereas patients gave them the lowest score. In contrast, the “quality of evidence” item scored lowest among patients and had the highest mean score among evaluators (Figs. S1A–S1E in Appendix A).

The overall value estimate integrates the weights and scores of each panelist on a scale from 0 to 100. Thus, Figs. 2 and 3 show the mean contribution of each criterion to the overall value estimate for hospital and outsourced dialysis, respectively. The estimate of the value of the outsourced dialysis was 0.29 (SD: ±0.2) and for hospital dialysis 0.39 (SD: ±0.2). When analyzing the mean score of the overall value, according to professional profile, it can be seen that all the profiles gave a higher value to hospital hemodialysis (range: 0.44–0.36 for patients and managers, respectively) than to conventional dialysis (range: 0.31–0.27 for patients and middle management, respectively) (Figs. S2A–S2E in Appendix B for hospital dialysis and Figs. S3A–S3E for conventional dialysis, according to professional profile).

Fig. 2.

Contributions of mean score of each quantitative criterion and estimates of the overall score of hospital dialysis.

Fig. 3.

Contributions of mean score of each quantitative criterion and estimates of the overall score of outsourced dialysis.


Finally, the workshop participants evaluated the impact of each qualitative or contextual criterion,11 with the criterion “common objective and specific interests” showing the greatest negative impact (12 participants rated it as such). On the other hand, “opportunity cost and affordability,” and “population priorities and access were the two criteria that showed the greatest positive impact” (Fig. 4).

Fig. 4.

Impact of the qualitative scores assigned by the participants in the evaluation.


MCDA is increasingly used worldwide for health care decisions to improve the consistency and transparency of policy decisions.14 MCDA applications can be classified into two categories12,15: 1) resource allocation, assigning priority based on the outcome of MCDA application16 (reimbursement and authorization decisions or budget allocation) and 2) ranking therapeutic alternatives for specific clinical problems, developing clinical guidelines or incorporating patient preferences in shared decision-making.

This article presents an assessment, using the MCDA methodology, of two non-exclusive alternatives in the field of the provision of treatment for a highly prevalent chronic disease whose assessment does not imply the allocation of resources toward one modality or the other. To the authors’ knowledge, there is little experience in the application of this methodology for decisions of this nature applied to the field of nephrology.

The complexity of the healthcare sectors, budgetary constraints, and the need to offer high quality services have led healthcare organizations to adopt new management approaches. Thus, outsourcing of services has been considered in recent years by health care decision-makers as one of the tools for improving organization, development, and productivity. The outsourcing of services is a mechanism for assigning some of the organization’s activities to an external provider. Among the various reasons for outsourcing are: cost reduction, increased efficiency by focusing on value-adding processes, improved skills, reduced service delivery time, and increased competitive advantage.17,18

The outsourcing of services is especially relevant in the treatment of chronic kidney disease; in Spain, public–private collaboration plays an important role in the dialysis provision model and many patients receive hemodialysis in out-of-hospital clinics, through agreements with the different health services.12

The prevalence of chronic kidney disease in Spain is around 15% and the results have shown that it can be considered a cardiovascular condition.19 Moreover, CKD represents a very high cost20: it accounts for 3% of total health expenditure and requires health care that involves a multitude of health professionals at different levels, with a high level of coordination for adequate prevention, diagnosis and treatment.21 Bearing in mind that we are situated in a setting characterized by the high cost to the health system of treating chronic renal failure, this work has approximated a measure that compares the value that two forms of service provision of dialysis through out-of-hospital clinics and hospitals can provide. A similar initiative developed a multicriteria evaluation methodology for dialysis centers based on the value of health care and understood as the ratio of patient benefit per monetary unit invested. Four criteria were used (evidence-based clinical performance, annual mortality, patient satisfaction, and health-related quality of life) and MCDA was used in the phase of assigning weights or weighting of criteria and sub-criteria. Although the number of external dialysis clinics included in the study was not high (n = 5), the three clinics that provided the highest value were all external.22

Regardless of the results, there is no doubt that a value approach, through MCDA or other similar methodology, has the potential to integrate different perceptions of all stakeholders and create a context for improving a treatment or the way in which a care service is provided, innovate processes, pay for performance, and improve transparency. Moreover, the results can be easily understood by the different stakeholders involved (by patients and their families, dialysis center health professionals, managers, providers, or political decision-makers) and are reproducible in different evaluation contexts.

At the end of the workshop, there was a space to collect the opinions of the participants on the applicability and utility of this methodology. In general, the panelists affirmed that it is a useful and interesting exercise and that, to a certain extent, it provides security in decision-making, since it allows different criteria considered to be ordered, justified, and explicitly considered. They also stated that the exchange of opinions among the participants enriched the process, especially because of the diversity of profiles.

These aspects are particularly relevant in the context of healthcare organizations, which are characterized by a high level of complexity and a very dynamic environment.23 Although the participants were offered a theoretical session on MCDA and an attempt was made to adapt the language to all the panelists, they reported difficulties with the terminology used, which required a certain familiarity, and uncertainty interpreting the evidence and the type of concepts used,24 especially among those with less experience in health technology assessment.

This work has several limitations that may affect the results of the study. First, due to the pandemic, the workshop was conducted virtually, which may have affected the interaction between participants and the discussion generated. On the other hand, the uncertainty and care overload generated during the pandemic made it impossible to carry out a subsequent retest to assess the consistency of the scores.25

The EVIDEM evaluation framework was used for this evaluation, but it is possible that it omitted certain criteria that might have been relevant such a specific evaluation as the one proposed in this work.26 Nevertheless, three professionals not participating in the workshop were asked for a prior individual evaluation in order to consider the adequacy of the criteria considered in the workshop.26 The EVIDEM framework was considered to cover all aspects relevant to the evaluation, did not require complicated calculations, and there was experience in its use.27

The relevant aspect of the result is not the absolute values obtained, but the comparison between the two values. Thus, the main contribution of the MCDA is the deliberative process itself and the discussion of the results.3,28 Thus, according to this study, the fact that professionals give more value to hospital dialysis than to conventional dialysis should translate into a prioritization in the allocation of health resources, decision-making and formulation of health policies.


The Andalusian School of Public Health (Escuela Andaluza de Salud Pública), the institution responsible for the management and development of the project, received funding from Vifor Pharma. Vifor Pharma had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Conflict of interest

The laboratory that funded this work did not participate or intervene in the group’s decisions including the choice of the problem, selection of workshop participants, or in the analysis, writing, presentation, and approval of the manuscript. The authors of this article declare that they are not subject to any conflict of interest related to the subject that may affect the design, analysis, or presentation of results.


To Marta Trapero Beltrán of the International University of Catalonia, David Epstein of the Faculty of Economics of the University of Granada, and Jaime Espín Balbino and Camila Higueras Callejón of the Andalusian School of Public Health.

Appendix A
Supplementary data

The following is Supplementary data to this article:

G. Atienza Merino, L. Varela Lema.
Needs and demands of policy-makers.
Health technology assessment and health policy making in Europe, pp. 137-160
L. Vuorenkoski, H. Toiviainen, E. Hemminki.
Decision-making in priority setting for medicines—a review of empirical studies.
N. Zozaya, J.O. Moreno, ÁH. Vega.
El análisis de decisión multi-criterio en el ámbito sanitario: utilidad y limitaciones para la toma de decisiones.
Fundación Weber, (2018),
P. Wahlster, M. Goetghebeur, C. Kriza, C. Niederländer, P. Kolominsky-Rabas.
Balancing costs and benefits at different stages of medical innovation: a systematic review of multi-criteria decision analysis (MCDA).
BMC Health Serv Res, 15 (2015), pp. 262
K. Marsh, J.J. Caro, A. Hamed, E. Zaiser.
Amplifying each patient’s voice: a systematic review of multi-criteria decision analyses involving patients.
Appl Health Econ Health Policy, 15 (2017), pp. 155-162
S. Márquez-Peláez, J. Espín Balbino, A. Olry de Labry Lima, V. Benítez Hidalgo, grupo de trabajo Redets para MCDA.
Guía para la elaboración de recomendaciones basada en análisis de decisión multicriterio. Sevilla: AETSA, Evaluación de Tecnologías Sanitarias de Andalucía.
Red Española de Agencias de Evaluación de Tecnologías Sanitarias, (2020),
P. Thokala, A. Duenas.
Multiple criteria decision analysis for health technology assessment.
Value Health, 15 (2012), pp. 1172-1181
H.E. Castro Jaramillo, M. Goetghebeur, O. Moreno-Mattar.
Testing multi-criteria decision analysis for more transparent resource-allocation decision making in Colombia.
Int J Technol Assess Health Care, 32 (2016), pp. 307-314
R. Baltussen, K. Marsh, P. Thokala, V. Diaby, H. Castro, I. Cleemput, et al.
Multicriteria decision analysis to support health technology assessment agencies: benefits, limitations, and the way forward.
Value Health, 22 (2019), pp. 1283-1288
T.D.C. Frazão, D.G.G. Camilo, E.L.S. Cabral, R.P. Souza.
Multicriteria decision analysis (MCDA) in health care: a systematic review of the main characteristics and methodological steps.
BMC Med Informa Decis Mak, 18 (2018), pp. 90
M.M. Goetghebeur, M. Wagner, H. Khoury, R.J. Levitt, L.J. Erickson, D. Rindress.
Bridging health technology assessment (HTA) and efficient health care decision making with multicriteria decision analysis (MCDA): applying the EVIDEM framework to medicines appraisal.
Med Decis Making, 32 (2012), pp. 376-388
A. Caro Martínez, M.Á González Vera, M. Prieto Velasco, A. Olry de Labry Lima.
Evidencia sobre la externalización de los servicios de diálisis: Una revisión de alcance.
A. Moreno-Calderón, T.S. Tong, P. Thokala.
Multi-criteria decision analysis software in healthcare priority setting: a systematic review.
PharmacoEconomics, 38 (2020), pp. 269-283
P. Thokala, N. Devlin, K. Marsh, R. Baltussen, M. Boysen, Z. Kalo, et al.
Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force.
Value Health, 19 (2016), pp. 1-13
K. Marsh, M. IJzerman, P. Thokala, R. Baltussen, M. Boysen, Z. Kaló, et al.
Multiple criteria decision analysis for health care decision making—emerging good practices: report 2 of the ISPOR MCDA Emerging Good Practices Task Force.
Value Health, 19 (2016), pp. 125-137
M.N. Farghaly, S.A.M. Al Dallal, A.N. Fasseeh, N.A. Monsef, E.A. Suliman, M.A. Tahoun, et al.
Recommendation for a pilot MCDA tool to support the value-based purchasing of generic medicines in the UAE.
Front Pharmacol, 12 (2021), pp. 680737
Z. Kavosi, H. Rahimi, S. Khanian, P. Farhadi, E. Kharazmi.
Factors influencing decision making for healthcare services outsourcing: a review and Delphi study.
Med J Islam Repub Iran, 32 (2018), pp. 56
A.M. Bach-Mortensen, J. Barlow.
Outsourced austerity or improved services? A systematic review and thematic synthesis of the experiences of social care providers and commissioners in quasi-markets.
Soc Sci Med, 276 (2021), pp. 113844
M. Gorostidi, M. Sánchez-Martínez, L.M. Ruilope, A. Graciani, J.J. de la Cruz, R. Santamaría, et al.
Chronic kidney disease in Spain: prevalence and impact of accumulation of cardiovascular risk factors.
Nefrologia, 38 (2018), pp. 606-615
S. Márquez-Peláez, A. Caro-Martínez, D. Adam-Blanco, A. Olry-de-Labry-Lima, J.A. Navarro-Caballero, L. García-Mochón, et al.
Eficiencia de la diálisis peritoneal frente a hemodiálisis para el tratamiento de la insuficiencia renal. Agencia de Evaluación de Tecnologías Sanitarias. Escuela Andaluza de Salud Pública. Consejería de Igualdad, Salud y Políticas Sociales, Junta de Andalucía.
Documento marco sobre enfermedad renal crónica (ERC) dentro de la Estrategia de Abordaje a la Cronicidad en el SNS [Accessed 28 April 2022]. Available from:
E. Parra, M.D. Arenas, M. Alonso, M.F. Martínez, Á. Gamen, J. Aguarón, et al.
Assessing value-based health care delivery for haemodialysis.
J Eval Clin Pract, 23 (2017), pp. 477-485
K. Kidholm, A.M. Ølholm, M. Birk-Olsen, et al.
Hospital managers’ need for information in decision making—an interview study in nine European countries.
Health Policy Amst Neth, 119 (2015), pp. 1424-1432
T. Shafaghat, M.H. Imani Nasab, M.A. Bahrami, Z. Kavosi, M. Roozrokh Arshadi Montazer, M.K. Rahimi Zarchi, et al.
A mapping of facilitators and barriers to evidence-based management in health systems: a scoping review study.
M. Gasol, N. Paco, L. Guarga, J.À Bosch, C. Pontes, M. Obach.
Early access to medicines: use of Multicriteria Decision Analysis (MCDA) as a decision tool in Catalonia (Spain).
J Clin Med, 11 (2022), pp. 1353
A. Dubromel, M.A. Duvinage-Vonesch, L. Geffroy, C. Dussart.
Organizational aspect in healthcare decision-making: a literature review.
J Mark Access Health Policy, 8 (2020), pp. 1810905
T. Shafaghat, M.H. Imani Nasab, M.A. Bahrami, Z. Kavosi, M. Roozrokh Arshadi Montazer, M.K. Rahimi Zarchi, et al.
A mapping of facilitators and barriers to evidence-based management in health systems: a scoping review study.
R. Baltussen, S. Youngkong, F. Paolucci, L. Niessen.
Multi-criteria decision analysis to prioritize health interventions: capitalizing on first experiences.
Health Policy, 96 (2010), pp. 262-264

The results of this work were presented at the XL Jornadas de Economía de la Salud (AES).

Copyright © 2022. Sociedad Española de Nefrología
Nefrología (English Edition)
Article options
Supplemental materials
es en

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?