Validation of EuroSCORE II Scoring System on Isolated CABG Patient in Indonesia

Abstract

Background

Coronary Artery Bypass Graft (CABG) is one solution to overcome cardiovascular problems. EuroSCORE II is a scoring system to predict mortality risk in patients undergoing cardiac surgery including CABG. Unfortunately, there’s still much debate about the benefits of EuroSCORE II in Asia, including Indonesia. This study aims to validates EuroSCORE II in predicting the outcomes in patients underwent CABG without any other procedure.

Results

A total of 2628 patients were included. The mean age was 59 years, mostly male (84.97%; n = 2233). Most patients underwent elective surgery (93.07%; n = 2446) and only 1.67% (n = 44) of the patients has high EuroSCORE category. Death was found in 4.22% (n-111) patients. EuroSCORE II had fair discriminant power (AUC 0.72), but a lower mortality predicted value for each group.

Conclusion

The parameters in EuroSCORE II are related with mortality in isolated CABG patients, but they cannot be used as mortality predictors in Indonesia.

Background

Cardiovascular disease is one of many health problems that causes significant death rate in the world. Around 31% of deaths are caused by cardiovascular disease worldwide, and in Indonesia it was one of three most death-causing non communicable diseases, besides stroke and cancer [1]. One of the solutions provided to overcome this problem is Coronary Artery Bypass Graft (CABG). This procedure is mainly indicated for patients who are at high risk of death, or have more than one vascular problem, or myocardial infarction that cannot be managed by Percutaneous Coronary Intervention (PCI) [2]. Although this procedure can reduce mortality, there is still a chance that the patient will have a poor outcome. European System for Cardiac Operative Risk Evaluation (EuroSCORE) is a scoring system to predict mortality risk in patients undergoing cardiac surgery, including CABG. This scoring system was published for the first time in 1999, and then updated in 2012 [3].

Several follow-up studies have been done to validate EuroSCORE II around the globe. Noyez, et.al stated that the parameters used in EuroSCORE II can indeed reduce the chance of over estimation of mortality risk that occurred in the previous scoring system [4]. Other meta-analysis studies conducted in the United States and England also provided the same conclusion, i.e., that EuroSCORE II have a higher predictive value and overpowering ACEF scoring system in terms of in-hospital mortality and 30-day mortality [5].

Unfortunately, there’s still much debate about the benefits of EuroSCORE II in Asia, including Indonesia. A study in Kuala Lumpur showed that EuroSCORE II is suitable enough to predict mortality after CABG procedures, but in India, EuroSCORE II was shown to have bad discrimination power and poor calibration [6, 7]. In Indonesia, EuroSCORE II is yet to be used as consideration for managing patients undergoing major cardiac surgery due to the minimum number of supporting studies. A cohort study conducted in Surabaya showed that EuroSCORE II has fair calibration and discrimination power, but recently another study also conducted in Surabaya stated that EuroSCORE II is less effective for use in Indonesia [8, 9]. A recent multi-center study also supports the statement that EuroSCORE II is a poor predictor for major cardiac surgery patient outcomes in Indonesia [10]. Isolated CABG is a cardiac procedure in which only CABG is performed without any other procedure such as valve repair, structural repair, aorta surgery, or tumor resection. Currently, there are no studies that focus on analyzing the benefit of EuroSCORE II in isolated CABG patient populations in Indonesia.

Methods

This research is a cross sectional retrospective study conducted at the Harapan Kita National Heart Center. Ethical clearance and research approval were granted by the director of Harapan Kita National Heart Center. Variables’ definitions such as COPD, poor mobility, endocarditis, previous cardiac surgery, kidney failure category, ejection fraction category, and procedural status were based on the original EuroSCORE II study. Subjects were taken by total sampling. All patients aged 18 years or older who underwent an isolated CABG procedure between January 2017 and June 2022 and had sufficient data for EuroSCORE II calculation were included in this study. Patient data were extracted from the registry of Adult Surgery Division, Research and Development Installation of Harapan Kita National Heart Center.

Data processing was performed using SPSS 17 software. Univariate analysis was performed on patient characteristic data. Categorical data are displayed in proportion or percentage, and numerical data in the form of mean and standard deviation (if normally distributed) or median and minimum–maximum (if not normally distributed). Discriminatory power and calibration tests were performed to assess the predictive performance of EuroSCORE II. The area under the receiver operating characteristics (ROC) curve was observed to estimate the discriminatory performance of EuroSCORE II in predicting in-hospital mortality. The Hosmer–Lemeshow goodness-of-fit test was performed to assess calibration.

Results

From January 2017 to June 2022, 2862 isolated CABG procedures were performed at Harapan Kita National Heart Center, in which a total of 2628 patients meet the inclusion criteria. From the patients included in the study, 75% (n = 2175) is in low EuroSCORE II category, 16% (n = 409) in moderate category and 2% (n = 44) in high category. Complete characteristics can be seen in Table 1. This study found that factors assessed by EuroSCORE II is in fact related with mortality in Indonesian isolated CABG patients as shown in Table 2.

Table 1 Sample characteristic

Table 2 Association  between risk factor and mortality

Based on the receiver operating characteristic (ROC) curve, it can be seen that EuroSCORE II has a fair discriminant power with an area under the curve (AUC) of 0.72 (Fig. 1). However, when reanalyzed with Hosmer–Lemeshow to determine calibration, EuroSCORE II was found to have poor calibration (p = 0.02), with a lower mortality predictive value in each risk category group (Table 3). The calibration plot also suggest the same result with slope value of 0.814 and CITL value of 0.009 (Fig. 2).

Fig. 1

ROC curve of EuroSCORE II discrimination on mortality prediction

Fig. 2

Calibration plot of EuroSCORE II

Table 3 Association between predicted mortality and observed mortality in isolated CABG patients

Discussion

Despite of fair discrimination power, current study suggests that EuroSCORE II underestimate mortality risk in all scoring category in this population. This result is supported by a multi-center study done by Kurniawaty, et.al conducted at Dr. Sardjito Hospital, Kariadi Hospital, and Abdul Wahab Sjahranie Hospital, which stated that EuroSCORE II had a poor predictive value and led to an underestimation of mortality risk in patients undergoing major cardiac surgery, though some parameters included in the scoring system did have significant association with patient outcomes. Diabetes, history of previous cardiac surgery, left ventricular dysfunction, history of myocardial infarction, and procedure status which each had a significant OR in this study were found to have no significant association in this multi-center study. It should be emphasized that these studies involved several types of cardiac surgical procedures, and the majority were non-coronary procedures [10].

In contrast, Sembiring, et.al found a slightly different results where EuroSCORE II had good calibration as a predictor for mortality in patients undergoing major cardiac surgery (including surgery, heart tumor surgery, aorta surgery, or a combination of these surgeries) based on the Hosmer–Lemeshow analysis (p = 0.55), and the area under the ROC curve is 0.85 which indicates a good discriminant value. However, this study actually overestimated mortality in EuroSCORE II. An additional finding obtained from that study was that this scoring system tends to overestimate the risk of death for the group of patients with EuroSCORE II < 1.3% and underestimate the risk of death for the group of patients with EuroSCORE II > 2.3% [8]. Another study that also used samples undergoing isolated CABG in Medan also found a statistically significant association with EuroSCORE II stratification on mortality. Moreover, EuroSCORE with a value of > 3.31 could be used to predict major cardiovascular events after CABG with a sensitivity of 90% and a specificity of 90%. Unfortunately, the study did not validate the accuracy of the use of EuroSCORE II itself [11].

The patient’s overall characteristics in this research were not significantly different from those of the original EuroSCORE II study. The only significant difference between the two studies was that endocarditis was not found in any of the patients in this study [3]. Patients with endocarditis are more likely to undergo concurrent valve surgery procedures; whereas, this study focused on patients undergoing CABG alone. Stroke (OR 2.28; 95% CI 1.331–3.903) and hypertension (OR 1.62; 95% CI 1.039–2.511) were found to significantly increase the risk of mortality in this study. These two comorbidities had not been considered in the original EuroSCORE II study, although Herlitz, et.al already suggested in their study that hypertension increases the risk for direct post-procedural complications and two year mortality rate [3, 12]. The effect of stroke on the increased risk of death in this study may because patients with a history of stroke also have poor mobility. Bottle, et.al found that history of stroke before CABG did not affect the outcomes of the procedure unless accompanied by other morbidities [13]. In this study, the onset of myocardial infarction was also found to influence the mortality risk in patients. In the original EuroSCORE II study, similar results were found with onset category of < 72 h, 72 h—three months, and more than three months. This parameter was not included in the final scoring system because it was considered to have the same value as the patient's procedure status, and when reanalyzed by the regression method, the association of onset with mortality risk was significantly reduced [3].

In Asia, the use of EuroSCORE II as a mortality predictor in post-cardiac surgery patients is still being debated. Studies in Malaysia and Bangladesh that were also conducted in isolated CABG populations found that EuroSCORE II was good enough to predict mortality in their research population. Musa, et.al found no significant difference between predicted mortality based on EuroSCORE II and observed mortality, which indicates a good model calibration [6]. Ranjan et.al who found that EuroSCORE have an important role in predicting early prognosis and end outcome also support the previous study, even though there’s a little discrepancy between the expected mortality and the observed mortality in their result [14]. In India, EuroSCORE II is considered not ideal for predicting mortality in post-major cardiac surgery patients according to a cohort study which found that the scoring system only correctly predicted the low and moderate risk patients, but overestimated the high risk group [7].

Research conducted in Greece, Serbia, the Netherlands, Argentina, as well as meta-analysis studies in America and the UK suggest that EuroSCORE II has improved the previous scoring system and is quite valid to predict mortality in post-heart surgery patients, either isolated CABG, valve surgery, or combined surgery [5, 15,16,17,18]. An interesting finding from a multi-center study in Argentina stated that EuroSCORE II showed adequate performance in terms of discrimination and calibration for all types of surgery, although it was somewhat lower for coronary surgery [15]. According to H.L. Blum’s theory, the degree of a person's health can be determined by 40% of environmental factors, 30% of behavioral factors, 20% of health care factors, and 10% of genetic factors [19]. In addition, there are several factors other than health services that may be the reason for the differences in the validity of EuroSCORE II in Indonesia and abroad.

There may be some possible limitations in this study. This study was conducted at a national heart center where many patients were referred by the smaller hospital and tend to have other comorbidity that could worsen patient’s outcome. It should be noted that in this study several variables were not taken into account in EuroSCORE II, but had significant associations with mortality such as the onset of infarction and hypertension. Inaccurate patient comorbid history measured in EuroSCORE II due to patient misunderstandings during history taking can also contribute to this difference since our data based on patient anamnesis when they first brought to the ER. For example, in this study, 3.6% of patients who were not diagnosed with diabetes had HbA1C > 6.5% with 53.4% missing HbA1C data. Whereas according to research by Zheng, et.al, HbA1c levels were potentially associated with an increased risk of all-cause death, myocardial infarction, and stroke in diabetic subjects undergoing CABG surgery [20]. Research by Soewondo, et.al which examined several data sources available in Indonesia also stated that the prevalence of diabetes was 5.7%, with 70% of cases being undiagnosed [21]. Further research is needed to establish a more accurate scoring system for the Asian population, especially Indonesia, by taking into account these variables and possibly other variables that have not been taken into account in this study.

Conclusions

Although all parameters measured in the EuroSCORE II scoring system have a significant association in increasing the risk of mortality in isolated CABG patients, the EuroSCORE II scoring system still cannot be used as an accurate predictor for the Indonesian population.