J Neurointensive Care Search

CLOSE


J Neurointensive Care > Volume 6(2); 2023 > Article
Cheon, Kwon, and Ryu: Evaluating the Prognostic Efficacy of Scoring Systems in Neurocritical and Neurosurgical Care: An Insight into APACHE II, SOFA, and GCS

Abstract

Background

To investigate the utility of established prognostic scoring systems, such as the Acute Physiology and Chronic Health Evaluation (APACHE) II score, Sequential Organ Failure Assessment (SOFA) score, and Glasgow Coma Scale (GCS), for patients admitted to a neurosurgical intensive care unit (ICU).

Methods

Among neurosurgical patients admitted to the neurosurgical ICU in a tertiary hospital from January 2015 and December 2022, only patients who had an ICU stay exceeding three days were included. The primary endpoint was in-hospital mortality.

Results

In this study, a total of 3,417 patients were enrolled in the study. Of these, 3,052 (89.3%) survived until hospital discharge. Both the APACHE II and SOFA scores were significantly higher in non-survivors than in survivors (p<0.001 for both). Conversely, GCS and GCS motor score (GCS M) were substantially lower in non-survivors (p<0.001). Among the commonly used scoring systems, the APACHE II score emerged as the most effective predictor of in-hospital mortality (C-statistic of 0.887, 95% confidence interval: 0.869–0.887). Remarkably, the GCS M proved to be equally effective as the SOFA score in predicting in-hospital mortality (p=0.435) and offered the additional advantage of being simpler to use.

Conclusion

Our findings indicate that these scoring systems offer valuable insights into the clinical prognosis of patients in the neurosurgical ICU. Moreover, the GCS M stands out as a feasible and reliable metric for predicting in-hospital mortality among neurosurgical ICU patients.

INTRODUCTION

The prognosis of critically ill patients remains an intricate yet crucial component in the management and decision-making processes within intensive care units (ICU)1,2). Several scoring systems like the Acute Physiology and Chronic Health Evaluation (APACHE) II score, Sequential Organ Failure Assessment (SOFA) score, and Glasgow Coma Scale (GCS) are often utilized to gauge the severity of a patient's condition and to predict clinical outcomes3-5). While these scores have proven valuable in general ICUs, they are not without limitations—especially when applied to specialized areas like neurointensive care.
The landscape of neurointensive care presents unique challenges and complexities, often demanding a specialized approach to both management and prognosis6). Neurocritically ill patients often suffer from a wide array of neurological injuries and diseases, ranging from traumatic brain injuries to strokes, each with distinct pathophysiologies and prognostic indicators7,8). Despite the growing body of research in critical care medicine, there is currently a paucity of scoring systems that are specifically tailored for predicting outcomes in neurointensive care settings. In addition, there is a limited body of research investigating the efficacy of established scoring systems like APACHE II score, SOFA score, and GCS in predicting outcomes specifically for patients in neurointensive care settings. Generally, these prognostic scoring systems have been commonly used for predicting clinical outcomes in critically ill patients. Therefore, the aim of this study was to assess the utility of established prognostic scoring systems, such as the APACHE II score, SOFA score, and GCS, for patients admitted to a neurosurgical ICU.

METHODS

This was a retrospective, single-center, observational study conducted on patients admitted to the neurosurgical intensive care unit (ICU) of a tertiary referral hospital (Samsung Medical Center, Seoul, Korea) between January 2015 and December 2022. The study received approval from the Institutional Review Board (IRB) of Samsung Medical Center (No. SMC 2020-09-082) and was carried out in compliance with the principles of the Declaration of Helsinki. Given the retrospective design of the study, the IRB waived the requirement for informed consent. Our study population consisted of patients who had been admitted to the Neurosurgical ICU for the management of neurocritical illnesses or for postoperative care following neurosurgical procedures. Furthermore, only patients who had an ICU stay exceeding three days were included. We excluded patients with incomplete medical records, those with 'Do Not Resuscitate' orders, those admitted to departments other than Neurosurgery, or those transferred to other facilities with unknown prognoses (Fig. 1).

Definitions and endpoints

In this study, we retrospectively collected baseline characteristics including comorbidities, behavioral risk factors, ICU management strategies, and laboratory data through our Clinical Data Warehouse, known as "Darwin-C." This platform was specifically designed to enable researchers to search and retrieve de-identified medical records from electronic archives. In this study, results of blood laboratory tests as well as APACHE II score, SOFA score, and GCS were automatically extracted from the medical records. The APACHE II score was calculated based on initial values from 12 routine physiological measurements, age, and pre-existing health conditions to provide a comprehensive measure of disease severity9). An increasing score, ranging from 0 to 71, was strongly correlated with the risk of subsequent in-hospital mortality9). The SOFA score was determined by evaluating individual components related to respiratory, coagulation, liver, cardiovascular, central nervous system, and renal functions, as described in a prior study10). Both the APACHE II and SOFA scores were computed using the worst values recorded during the initial 24 hours following ICU admission. For intubated patients, the verbal component of the GCS was estimated using eye and motor scores, in accordance with previously established methods11). Invasive intracranial pressure (ICP) monitoring was performed by neurosurgeons. The decision to initiate ICP monitoring was primarily based on brain imaging findings and the patient’s clinical condition. Specifically, the presence of space-occupying lesions, such as brain tumors, abscesses, and intracranial hemorrhages (including epidural hematoma, subdural hematoma, and intraparenchymal hematoma), warranted monitoring when increased ICP was suspected. In cases presenting with cerebrospinal fluid flow obstruction due to hydrocephalus, an external ventricular drain was predominantly used. Furthermore, in instances of cerebral edema or when clinically suspected increased ICP was evident, invasive ICP monitoring was employed, contingent on the neurosurgeon's assessment, to facilitate neuromonitoring. The primary endpoint of this investigation was in-hospital mortality.

Statistical analysis

Continuous variables are presented as means with standard deviations, while categorical variables are shown as counts and corresponding percentages. Statistical comparisons were made using Student's t-test for continuous variables and either the chi-square test or Fisher's exact test for categorical ones. The predictive performance of each scoring system was assessed through the areas under the receiver operating characteristic (ROC) curves, with sensitivity plotted against 1-specificity. The areas under the curves (AUCs) were compared using DeLong et al.'s nonparametric approach for evaluating two correlated AUCs12). Clinically relevant variables—such as severity scoring systems, age, sex, comorbidities, behavioral risk factors, reasons for ICU admission, and ICU management practices—were subjected to multiple logistic regression analysis to identify statistically significant predictors. The adequacy of the prediction model was evaluated using the Hosmer-Lemeshow test, as well as the AUCs. All statistical tests were two-sided, and a p-value of less than 0.05 was considered statistically significant. Data analysis was conducted using R Statistical Software (version 4.2.1; R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

Baseline Characteristics and clinical outcome

A total of 3,417 patients were enrolled in the study. Of these, 3,052 (89.3%) survived until hospital discharge. The median age of the patients was 55.7 ± 17.0 years, and 1,779 (52.1%) were male. The most prevalent comorbidities among the study population were malignancy (72.1%) and cerebrovascular disease (31.0%). The leading cause of ICU admission was brain tumor, accounting for 62.1% of cases. Among patients with brain tumors, the most common reasons for ICU admission were post-operative care (39.5%) and neuromonitoring (17.2%). Interventions such as mechanical ventilation, intracranial pressure monitoring, the use of multiple hyperosmolar agents, and vasopressor and inotropic agent administration were more frequently observed in non-survivors. A comparative analysis of the baseline characteristics between survivors and non-survivors is presented in Table 1.

Relationship between APACHE II, SOFA, and GCS

A comparison of severity scoring systems between survivors and non-survivors is presented in Table 2. Both the APACHE II and SOFA scores were significantly higher in non-survivors than in survivors (p<0.001 for both). Conversely, GCS and GCS M were substantially lower in non-survivors (p<0.001). Multivariable analysis revealed that the APACHE II score (adjusted odds ratio [OR]: 1.19, 95% confidence interval [CI]: 1.15–1.23), SOFA score (adjusted OR: 1.13, 95% CI: 1.07–1.19), and GCS M (adjusted OR: 0.72, 95% CI: 0.64–0.81) were significant predictors of in-hospital mortality. Other variables, including malignancy (adjusted OR: 1.47, 95% CI: 1.09–2.01), mechanical ventilation (adjusted OR: 1.92, 95% CI: 1.33–2.80), continuous renal replacement therapy (adjusted OR: 2.79, 95% CI: 1.38–5.61), and ICP monitoring (adjusted OR: 0.61, 95% CI: 0.44–0.84) were also significantly associated with in-hospital mortality. The model demonstrated good fit with a Hosmer–Lemeshow Chi-squared value of 6.135 (df=8, p=0.632) and an AUC of 0.906 (95% CI 0.890–0.923) (Table 3).
In the ROC curve analysis for predicting in-hospital mortality, the APACHE II score demonstrated superior performance with a C-statistic of 0.887 (95% CI: 0.869–0.887), which was greater than that of the SOFA score (C-statistic: 0.783, 95% CI: 0.757–0.783), GCS (C-statistic: 0.833, 95% CI: 0.808–0.833), and GCS M (C-statistic: 0.796, 95% CI: 0.769–0.796) (all p<0.01). Moreover, the AUC for GCS was greater than those for the SOFA score and GCS M (both p<0.01). However, there was no significant difference in predictive performance between the SOFA score and GCS M (p=0.435) (Fig. 2).

DISCUSSION

In this study, we evaluated the utility of widely recognized prognostic scoring systems—namely, the APACHE II score, SOFA score, and GCS—in neurocritically ill and neurosurgical patients. Our major findings can be summarized as follows: First, statistically significant differences were observed in the APACHE II, SOFA, GCS, and GCS M scores between survivors and non-survivors. Second, multivariable analysis identified several variables, including APACHE II score, SOFA score, GCS M, malignancy, mechanical ventilation, continuous renal replacement therapy, and ICP monitoring, as significantly associated with in-hospital mortality. Among the commonly used scoring systems, the APACHE II score emerged as the most effective predictor of in-hospital mortality. Remarkably, the GCS M proved to be equally effective as the SOFA score in predicting in-hospital mortality and offered the additional advantage of being simpler to use. In conclusion, all the evaluated scoring systems, including the APACHE II score, SOFA score, GCS, and GCS M, demonstrated utility in predicting clinical outcomes in patients admitted to a neurosurgical ICU.
In the complex and dynamic environment of the ICU, the utilization of validated scoring systems like the APACHE II and the SOFA score becomes imperative for several reasons. These scoring systems provide healthcare providers with a structured, evidence-based framework to assess the severity of illness, enabling timely and appropriate medical interventions13). By offering a quantitative measure of disease severity, these scores help in stratifying patients based on risk, thereby aiding in the allocation of vital resources and guiding clinical decision-making14,15). Additionally, they facilitate more transparent and data-driven discussions with family members about the potential prognosis and assist in cost-benefit analyses14,15). Importantly, they can also serve as valuable research tools, providing a standardized measure of patient severity that allows for meaningful comparisons across studies and settings. Despite their limitations and the need for periodic updates and revisions, the centrality of such scoring systems in optimizing patient outcomes in the ICU cannot be overstated. Generally, prognostic scoring systems are invaluable for assessing disease severity, conducting cost-benefit analyses, and informing clinical decision-making3,5). However, many of these systems are complicated and challenging to implement, particularly in critically ill patients16). Given these limitations, there is a need for simplified yet reliable scoring systems. In this context, the GCS M score emerges as a feasible and reliable metric for predicting in-hospital mortality among neurocritically ill and neurosurgical patients17).
The APACHE II score may have advantages over other commonly used indices like the SOFA score or the GCS in certain contexts9). APACHE II not only incorporates a wide range of physiological variables but also considers age and pre-existing comorbidities, thus providing a comprehensive overview of a patient's condition4). This multidimensional approach potentially allows for a more nuanced risk stratification of ICU patients, which can be crucial for tailoring treatment strategies and resource allocation. Additionally, APACHE II's more extensive set of criteria might offer higher predictive validity for a broader spectrum of diseases and complications, making it a more versatile tool in diverse clinical settings18). While all of these scoring systems serve the essential function of aiding prognosis and guiding treatment, the unique features of APACHE II may render it particularly effective in capturing the complex interplay of factors that determine outcomes in critically ill patients. Therefore, in this study, the APACHE II score demonstrated greater predictive accuracy for clinical prognosis of patients in the neurosurgical ICU19).
This study has several limitations. First, this was a retrospective review of medical records and data extracted from the Clinical Data Warehouse. The nonrandomized nature of registry data might have resulted in a selection bias. Second, in measuring the GCS, we estimated the verbal score for intubated patients based on their eye and motor scores, following the methodology used in previous studies11). However, it should be acknowledged that this approach may not be entirely flawless. Third, in our study, unlike other ICU research, there was a disproportionately high prevalence of patients with malignancy and brain tumors. Although the present study provides valuable insights, prospective large-scale studies are needed to further confirm the usefulness of severity scoring systems in predicting clinical outcomes of neurocritically ill patients with evidence-based conclusions.

CONCLUSION

In this study, we explored the utility of well-established prognostic scoring systems, including the APACHE II score, SOFA score, and GCS, for assessing outcomes in neurocritically ill and neurosurgical patients. Our findings indicate that these scoring systems offer valuable insights into the clinical prognosis of patients in the neurosurgical ICU. Moreover, the GCS M stands out as a feasible and reliable metric for predicting in-hospital mortality among this patient population.

NOTES

Ethics statement

The study received approval from the Institutional Review Board (IRB) of Samsung Medical Center (No. SMC 2020-09-082) and was carried out in compliance with the principles of the Declaration of Helsinki. Given the retrospective design of the study, the IRB waived the requirement for informed consent.

Author contributions

Conceptualization: JAR. Data curation, Writing – original draft, Writing – review & editing: SC, JAR. Formal analysis, Statistical analysis: SC, HK.

Conflict of interest

There is no conflict of interest to disclose.

Funding

None.

Data availability

Regarding data availability, our data are available on the Harvard Dataverse Network (http://dx.doi.org/10.7910/DVN/ZSFPUY) as recommended.

Acknowledgements

We would like to express our gratitude to Suk Kyung Choo, the nursing director of the neurosurgical intensive care unit, for her valuable advice and insightful discussions. Additionally, we extend our thanks to all the nurses in the neurosurgical intensive care unit at Samsung Medical Center for their exceptional work.

Fig. 1.
Flowchart of the study.
ICU: intensive care unit.
jnic-2023-00696f1.jpg
Fig. 2.
Receiver operating characteristic curves for prediction of in-hospital mortality using Acute Physiology and Chronic Health Evaluation (APACHE) II score, Sequential Organ Failure Assessment (SOFA) score, Glasgow Coma Scale (GCS) and motor score of GCS (GCS M). AUC: areas under the curve, CI: confidence interval.
jnic-2023-00696f2.jpg
Table 1.
Baseline characteristics according to in-hospital mortality
Survivor (n = 3,052) Non-survivor (n = 365) P value
Patient demographics
 Age (year) 55.2 (17.0) 60.7 (16.2) <0.001
 Sex, male 1587 (52.0) 192 (52.6) 0.871
Comorbidities
 Malignancy 2261 (74.1) 204 (55.9) <0.001
 Cerebrovascular disease 871 (28.5) 188 (51.5) <0.001
 Hypertension 423 (13.9) 93 (25.5) <0.001
 Dyslipidemia 303 (9.9) 66 (18.1) <0.001
 Diabetes mellitus 230 (7.5) 56 (15.3) <0.001
 Chronic kidney disease 70 (2.3) 26 (7.1) <0.001
 Chronic liver disease 52 (1.7) 16 (4.4) 0.001
 Cardiovascular disease 45 (1.5) 12 (3.3) 0.019
Behavioral risk factors
 Current alcohol consumption 591 (19.4) 70 (19.2) 0.988
 Current smoking 308 (10.1) 34 (9.3) 0.708
Primary disease of ICU admission <0.001
 Brain tumor 1983 (65.0) 140 (38.4)
  Malignant brain tumor 733 (24.0) 29 (7.9)
  Benign brain tumor 494 (16.2) 17 (4.7)
  Brain metastasis 704 (23.1) 76 (20.8)
  Hematolymphoid tumor 52 (1.7) 18 (4.9)
 Intracerebral hemorrhage 217 (7.1) 61 (16.7)
 Traumatic brain injury 193 (6.3) 60 (16.4)
 Subarachnoid hemorrhage 201 (6.6) 43 (11.8)
 Elective vascular surgery 172 (5.6) 14 (3.8)
 Cerebral infarction 99 (3.2) 12 (3.3)
 Central nervous system infection 45 (1.5) 13 (3.6)
 Epilepsy 28 (0.9) 1 (0.3)
 Others 114 (3.7) 21 (5.8)
Cause of ICU admission 0.036
 Post-operative management 1955 (64.1) 202 (55.3)
 Neuromonitoring 846 (27.7) 129 (35.3)
 Respiratory failure 61 (2.0) 10 (2.7)
 Pre-operative management 27 (0.9) 4 (1.1)
 Cardiovascular disease 14 (0.5) 2 (0.5)
 Postcardiac arrest syndrome 14 (0.5) 0 (0.0)
 Post-procedural management 11 (0.4) 3 (0.8)
 Others 124 (4.1) 15 (4.1)
ICU management
 Mechanical ventilation 1045 (34.2) 305 (83.6) <0.001
 Continuous renal replacement therapy 29 (1.0) 28 (7.7) <0.001
 ICP monitoring 745 (24.4) 107 (29.3) 0.047
 Use of mannitol* 2294 (75.2) 248 (67.9) 0.003
 Use of glycerin* 825 (27.0) 188 (51.5) <0.001
 Use of more than one hyperosmolar agent 2309 (75.7) 258 (70.7) 0.044
 Use of vasopressors and inotropic agent 24 (0.8) 21 (5.8) <0.001

Data are presented as numbers (%) or means ± standard deviations.

* Some patients received more than one hyperosmolar agent.

ICU: Intensive care unit, ICP:, Intracranial pressure.

Table 2.
Severity scoring systems between survivors and non-survivors
Survivor Non-survivor p value
APACHE II score 11.22 ± 5.22 21.70 ± 6.78 <0.001
SOFA score 1.97 ± 2.18 5.10 ± 3.38 <0.001
GCS 13.42 ± 2.64 8.05 ± 4.43 <0.001
GCS M 5.66 ± 0.88 3.61 ± 1.94 <0.001

Data are presented as means ± standard deviations.

APACHE II:Acute Physiology and Chronic Health Evaluation, SOFA: Sequential Organ Failure Assessment, GCS: Glasgow Coma Scale, GCS M: motor score of GCS.

Table 3.
Predicting factors for in-hospital mortality in neurocritically ill patients and neurosurgical patients assessed using logistic regression model
Adjusted Odds Ratioa (95% CI) p value
APACHE II score 1.19 (1.15 – 1.23) <0.001
SOFA score 1.13 (1.07 – 1.19) <0.001
GCS M 0.72 (0.64 – 0.81) <0.001
Malignancy 1.47 (1.09 – 2.01) 0.014
Mechanical ventilation 1.92 (1.33 – 2.80) 0.001
Continuous renal replacement therapy 2.79 (1.38 – 5.61) 0.004
ICP monitoring 0.61 (0.44 – 0.84) 0.003

a Adjusted for severity scoring systems, age, sex, comorbidities, habitual risk factors, causes of intensive care unit (ICU) admission, and ICU management.

CI: Confidence interval, APACHE: Acute Physiology and Chronic Health Evaluation, SOFA: Sequential Organ Failure Assessment, GCS: Glasgow Coma Scale, GCS M: Motor score of GCS, ICP: Intracranial pressure.

REFERENCES

1. Keegan MT, Soares M. What every intensivist should know about prognostic scoring systems and risk-adjusted mortality. Rev Bras Ter Intensiva 2016;28:264–269.
crossref pmid pmc
2. Ridley SA. Uncertainty and scoring systems. Anaesthesia 2002;57:761–767.
crossref pmid pdf
3. Bouch DC, Thompson JP. Severity scoring systems in the critically ill. Continuing Education in Anaesthesia Critical Care & Pain 2008;8:181–185.
crossref
4. Ferreira FL, Bota DP, Bross A, Mélot C, Vincent JL. Serial evaluation of the SOFA score to predict outcome in critically ill patients. Jama 2001;286:1754–1758.
crossref pmid
5. Unertl K, Kottler BM. [Prognostic scores in intensive care]. Anaesthesist 1997;46:471–480.
pmid
6. Newman-Toker DE, Perry JJ. Acute diagnostic neurology: challenges and opportunities. Acad Emerg Med 2015;22:357–361.
crossref pmid
7. Reis C, Wang Y, Akyol O, Ho WM, Ii RA, Stier G, et al. What's new in traumatic brain injury: update on tracking, monitoring and treatment. Int J Mol Sci 2015;16:11903–11965.
crossref pmid pmc
8. Ziaka M, Exadaktylos A. The heart is at risk: understanding stroke-heart-brain interactions with focus on neurogenic stress cardiomyopathy-a review. J Stroke 2023;25:39–54.
crossref pmid pmc pdf
9. Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med 1985;13:818–829.
crossref pmid
10. Vincent J-L, Moreno R, Takala J, Willatts S, De Mendonça A, Bruining H, et al. Working group on sepsis-related problems of the European Society of Intensive Care Medicine. The SOFA (sepsis-related organ failure assessment) score to describe organ dysfunction/failure. Intensive Care Med 1996;22:707–710.
crossref pmid pdf
11. Meredith W, Rutledge R, Fakhry SM, Emery S, Kromhout-Schiro S. The conundrum of the glasgow coma scale in intubated patients: a linear regression prediction of the glasgow verbal score from the glasgow eye and motor scores. J Trauma 1998;44:839–844; discussion 844-835.
pmid
12. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837–845.
crossref pmid
13. Kim YH, Yeo JH, Kang MJ, Lee JH, Cho KW, Hwang S, et al. Performance assessment of the SOFA, APACHE II scoring system, and SAPS II in intensive care unit organophosphate poisoned patients. J Korean Med Sci 2013;28:1822–1826.
crossref pmid pmc pdf
14. Jang JH, Hong S, Ryu JA. Prognostic value of c-reactive protein and albumin in neurocritically ill patients with acute stroke. J Clin Med 2022;11:5067.
crossref pmid pmc
15. Kim KS, Oh AR, Park J, Ryu JA. Association between Fibrinogen-to-Albumin Ratio and Prognosis in Patients Admitted to an Intensive Care Unit. J Clin Med 2023;12:1407.
crossref pmid pmc
16. Pellathy TP, Pinsky MR, Hravnak M. Intensive care unit scoring systems. Crit Care Nurse 2021;41:54–64.
crossref pmc pdf
17. Majdan M, Steyerberg EW, Nieboer D, Mauritz W, Rusnak M, Lingsma HF. Glasgow coma scale motor score and pupillary reaction to predict six-month mortality in patients with traumatic brain injury: comparison of field and admission assessment. J Neurotrauma 2015;32:101–108.
crossref pmid pmc
18. Mumtaz H, Ejaz MK, Tayyab M, Vohra LI, Sapkota S, Hasan M, et al. APACHE scoring as an indicator of mortality rate in ICU patients: a cohort study. Ann Med Surg (Lond) 2023;85:416–421.
crossref pmid pmc
19. Bian Y, Zhang P, Xiong Yj, Xu F, Zhu S, Tang Z, et al. Application of the APACHE II score to assess the condition of patients with critical neurological diseases. Acta Neurologica Belgica 2015;115:651–656.
crossref pmid pdf
TOOLS
Share :
Facebook Twitter Linked In Google+ Line it
METRICS Graph View
  • 0 Crossref
  •    
  • 973 View
  • 20 Download
Related articles in JNIC


ABOUT
ARTICLE CATEGORY

Browse all articles >

BROWSE ARTICLES
AUTHOR INFORMATION
Editorial Office
Department of Neurosurgery, Korea University College of Medicine
73, Inchon-ro, Seongbuk-gu, Seoul 02841, Korea
Tel: +82-2-920-6833    Fax: +82-2-929-0629    E-mail: jnic.editor@gmail.com                

Copyright © 2024 by Korean Neurointensive Care Society.

Developed in M2PI

Close layer
prev next