Systemic Inflammatory Response Syndrome Criteria in Defining Severe Sepsis

Study Design

We conducted a retrospective study from January 1, 2000, to December 31, 2013, using data from the Australia and New Zealand Intensive Care Society (ANZICS) Adult Patient Database (APD),¹⁶ a high-quality database run by the ANZICS Centre for Outcome and Resource Evaluation. The ANZICS APD includes information on more than 90% of all ICU admissions in Australia and New Zealand. The Alfred Hospital Human Research Ethics Committee, Melbourne, Australia, approved the study with a waiver of informed consent. The data were gathered as a part of routine quality-assurance benchmarking processes by means of clinical registry surveillance by collectors in the participating ICUs.

Definitions

We defined sepsis, severe sepsis, and septic shock according to the American College of Chest Physicians–Society of Critical Care Medicine consensus definition.³ We used infection-related diagnoses according to the Acute Physiology and Chronic Health Evaluation (APACHE) III at admission to infer the presence of suspected or proven infection (see the Supplementary Appendix, available with the full text of this article at NEJM.org). We defined organ failure in the first 24 hours after ICU admission as a Sequential Organ Failure Assessment (SOFA) score of 3 or higher (on a scale from 0 to 4, with higher scores indicating more severe organ failure)^2,17,18 (see the Supplementary Appendix).

We diagnosed severe sepsis if a patient had one of the ANZICS APD diagnostic codes for infection and organ failure or one of the following prespecified additional diagnostic categories in the ANZICS APD diagnostic codes for infection: sepsis due to infection other than from the urinary tract with organ failure, sepsis due to urinary tract infection with organ failure, sepsis with shock due to infection other than from the urinary tract, or sepsis with shock due to urinary tract infection. We applied the consensus SIRS criteria to all the data analyses (see the Supplementary Appendix).³

All the above criteria were assessed within the first 24 hours after ICU admission. Patient follow-up was available only for the duration of hospital stay, and the primary outcome was in-hospital mortality.

Study Populations

Patients with SIRS-positive severe sepsis fulfilled the following requirements: symptoms meeting two or more SIRS criteria, an APACHE III diagnosis of infection at admission with at least one organ failure, or an APACHE III diagnosis of severe sepsis or septic shock at admission. Patients with SIRS-negative severe sepsis fulfilled the following requirements: symptoms meeting fewer than two SIRS criteria, and an APACHE III diagnosis of infection at admission with at least one organ failure or an APACHE III diagnosis of severe sepsis or septic shock at admission.

A descriptive analysis was performed on four prespecified subgroups of patients: patients with septic shock, those receiving mechanical ventilation, those with acute renal failure, and those with an APACHE II score of more than 24 (on a scale from 0 to 71, with higher scores indicating more severe disease). An additional post hoc analysis was performed to compare differences between community-acquired sepsis and hospital-acquired sepsis as determined by the ICU admission source.

Statistical Analysis

Data are presented as numbers and percentages, means and standard deviations, medians and interquartile ranges, or proportions with 95% confidence intervals. Accordingly, chi-square tests for equal proportion, Student's t-test, or the Wilcoxon rank-sum test were used to test differences. No assumptions were made for missing data, with multivariable analysis performed on data from patients who had complete data only (see the Supplementary Appendix).

To identify independent differences at baseline that may exist between patients with SIRS-positive severe sepsis and those with SIRS-negative severe sepsis, we applied multivariable logistic regression to data from all the patients with severe sepsis with SIRS-positive status as the outcome (see the Supplementary Appendix). The multivariable model was developed with the use of the least absolute shrinkage and selection operator (LASSO) method, with variable inclusion determined by the Schwarz–Bayesian information criterion. The resulting prediction equation was then used to generate each patient's probability of being SIRS-positive.

To investigate the similarities of differences in hospital outcomes over time for all the patients with sepsis, logistic-regression models were used (see the Supplementary Appendix), with the Australian and New Zealand calibrated Risk of Death (ANZROD) model.¹⁹ This model has been shown to perform better than the APACHE III in Australia and New Zealand.^20,21

To further determine the predictive capacity of using two or more SIRS criteria to identify an increase in the risk of death, SIRS was considered first as a dichotomous variable (≥2 SIRS criteria vs. 0 to 1 SIRS criterion) and second as an ordinal variable from 0 to 4, reflecting the number of SIRS criteria met. Because the SIRS criteria are derived from components related to the severity of sepsis in the patient, we determined the predictive capacity of the SIRS criteria by adjusting for the severity markers without SIRS criteria, in conjunction with year, site of admission, and probability of being SIRS-positive. Data were randomly divided into two equal sets, with the first used for model development and the second used for model validation. Model discrimination was determined with the use of the area under the curve (AUC), whereas classification error was determined with the use of integrated discrimination improvement and net reclassification improvement with results reported specifically for the validation data set. Net reclassification improvement was determined first as category-free improvement and second stratified in quartiles of risk.

To determine whether predictors of death differed significantly between SIRS-positive sepsis and SIRS-negative sepsis, a multivariable logistic-regression model for mortality among all the patients with sepsis was developed with the use of the LASSO method, with variable inclusion determined by means of the Schwarz–Bayesian information criterion. Interactions between identified predictors of mortality and SIRS status were then applied to determine whether the nature of the relationship to mortality differed according to SIRS status.

All the data were analyzed with the use of SAS software, version 9.4 (SAS Institute). A two-sided P value of less than 0.01 was considered to indicate statistical significance.

Study Patients

Table 1. Table 1. Baseline Characteristics and Hospital Outcomes of Patients with Severe Sepsis, According to Status with Respect to Criteria for the Systemic Inflammatory Response Syndrome (SIRS).

We studied data from 1,171,797 patients in 172 ICUs. A total of 1,062,134 patients did not have sepsis, and 109,663 had infection and organ dysfunction. Of the patients with infection and organ dysfunction, 96,385 patients (87.9%) had SIRS-positive severe sepsis and 13,278 (12.1%) had SIRS-negative severe sepsis (Table 1). Patients with SIRS-positive severe sepsis were younger, were more severely ill, and had higher mortality than those with SIRS-negative severe sepsis. They were also more likely to have septic shock or acute kidney injury but less likely to have a surgical admission or to be discharged home (Table 1).

The annual proportions of patients with SIRS-positive severe sepsis or SIRS-negative severe sepsis among all ICU admissions are presented in Fig. S1 in the Supplementary Appendix, as are the annual proportions of patients with SIRS-positive severe sepsis or SIRS-negative severe sepsis. Risk factors for SIRS-positive status are presented in Table S1 in the Supplementary Appendix.

SIRS in Severe Sepsis

Table 2. Table 2. Distribution of Signs Meeting SIRS Criteria in Patients with Severe Sepsis, According to SIRS-Positive and SIRS-Negative Status.

The distribution of SIRS criteria is presented in Table 2. The most frequent SIRS criterion that was met in patients with SIRS-positive severe sepsis was an increased heart rate, followed by an increased respiratory rate or a low partial pressure of arterial carbon dioxide (PaCO₂) and an abnormal white-cell count. In patients with SIRS-negative severe sepsis, the most frequent single criterion that was met was an abnormal white-cell count, followed by an increased heart rate and increased respiratory rate or a low PaCO₂ (Table 2). Of the patients with SIRS-negative sepsis, 20% did not fulfill any SIRS criteria, and 80% fulfilled one SIRS criterion (Table 2). The distribution of baseline characteristics among patients with SIRS-negative severe sepsis, stratified according to the presence of zero or one SIRS criterion, is presented in Table S2 in the Supplementary Appendix. The patients with SIRS-positive severe sepsis were significantly more likely than those with SIRS-negative severe sepsis to present with community-acquired infection (38.1% vs. 28.1%, P<0.001) or hospital-acquired infection (28.0% vs. 18.3%, P<0.001) (Tables S1 and S3 in the Supplementary Appendix).

Outcomes

Figure 1. Figure 1. Mortality among Patients with Severe Sepsis, According to Status with Respect to Criteria for the Systemic Inflammatory Response Syndrome (SIRS).

Patients were categorized according to whether they had symptoms meeting two or more SIRS criteria (SIRS-positive sepsis) or symptoms meeting less than two SIRS criteria (SIRS-negative sepsis). Panel A shows the unadjusted annual mortality among patients in the two groups from 2000 through 2013, and Panel B shows the adjusted annual odds of death. The bars represent 95% confidence intervals.

Absolute mortality decreased similarly in the two groups (Figure 1). The rate of death in SIRS-positive patients decreased from 36.1% (in 829 of 2296 patients) in 2000 to 18.3% (in 2037 of 11,119 patients) in 2013 (P<0.001). Over the same period, in SIRS-negative patients, the rate of death decreased from 27.7% (in 100 of 361 patients) to 8.5% (in 112 of 1315 patients) (P<0.001). These changes represent an annual rate of absolute decrease of 1.3 percentage points in each group, and a reduction in relative risk of 49.3% (45.7 to 52.6%) and 66.5% (57.5 to 73.6%), respectively.

Table 3. Table 3. Annual Change in Hospital Outcomes from 2000 through 2013 and Between-Group Differences.

There was a very similar annual decline in adjusted mortality in the two groups (annual odds ratio in the SIRS-positive group, 0.96; 95% confidence interval [CI], 0.96 to 0.97; odds ratio in the SIRS-negative group, 0.96; 95% CI, 0.94 to 0.98; P<0.001 for both comparisons) (P=0.12 for between-group difference) (Figure 1 and Table 3). The proportion of patients discharged home increased in the same way in the two groups (Fig. S2 in the Supplementary Appendix). The adjusted annual odds for discharge home increased significantly in the SIRS-positive group (annual odds ratio, 1.02; 95% CI, 1.01 to 1.02; P<0.001) but not in the SIRS-negative group (annual odds ratio, 1.01; 95% CI, 0.99 to 1.02; P=0.29) (Table 3). However, there was no significant difference between the two groups in the adjusted rate of increase (P=0.17). The unadjusted rate of discharge to another hospital appeared to increase similarly in the two groups (Fig. S2 in the Supplementary Appendix). However, in the adjusted analysis, there was no significant change over time in either group (Table 3). The adjusted rate of discharge to a rehabilitation or long-term care facility increased significantly in each group (P<0.001 for both comparisons), but there was no significant between-group difference (P=0.20) (Table 3).

Prediction of Mortality

Figure 2. Figure 2. Mortality among Patients with Severe Sepsis, According to Number of SIRS Criteria Met.

The bars represent 95% confidence intervals.

When the analysis included a modified risk of death (with SIRS components removed), the probability of being SIRS-positive, year of admission, study site, and SIRS status (positive or negative) as covariates in the logistic-regression analysis, being SIRS-positive independently increased the risk of death by 26% (odds ratio, 1.26; 95% CI, 1.18 to 1.34; P<0.001). In a similar model that included the number of SIRS criteria from 0 to 4, a 13% linear increase in mortality was associated with each additional SIRS criterion (odds ratio for each additional criterion, 1.13; 95% CI, 1.11 to 1.15; P<0.001) (Table S4 in the Supplementary Appendix) without any transitional increase in risk when two criteria were met (Figure 2). When a comparison of discrimination and reclassification was performed between SIRS criteria as a binomial variable (≥2 criteria vs. <2 criteria) and SIRS criteria as a continuous variable (0 to 4 criteria), there were modest but significant improvements noted for the continuous model in the AUC, integrated discrimination improvement, and net reclassification improvement when patients were stratified into quartiles of risk (Table S4 in the Supplementary Appendix).

Prediction of Mortality According to SIRS-Positive or SIRS-Negative Status

There were several predictors of mortality among patients with SIRS-positive sepsis or SIRS-severe sepsis (Table S5 in the Supplementary Appendix). Of 16 identified predictors of mortality, the only variable that differed significantly between the two groups was illness severity (Table S5 in the Supplementary Appendix).

We studied the sensitivity, face validity, and construct validity of the rule of using two or more SIRS criteria for the diagnosis of severe sepsis in the first 24 hours after ICU admission. We found that the SIRS-criteria rule missed one patient in eight with severe sepsis. Such patients with SIRS-negative severe sepsis had lower but still substantial mortality, as compared with patients with SIRS-positive sepsis, and the incidence, proportion, and mortality decreased over time almost identically to the rates among patients with SIRS-positive sepsis. In addition, their unadjusted and adjusted discharge rates to a rehabilitation or long-term care facility were also similar, as were predictors of mortality. Finally, in the adjusted analysis, mortality increased linearly with each additional SIRS criterion from 0 to 4 without any transitional increase in risk at a threshold of two criteria.

SIRS criteria were described more than two decades ago, and the signs meeting these criteria have been assumed to indicate a clinical response to inflammation.³ Infection was seen to require the presence of such signs to help define the transition to sepsis, severe sepsis, and septic shock.^15,22,23 This approach was supported by the observation that an increase in the number of SIRS criteria met was associated with a worse outcome in critically ill patients, regardless of infection status.^15,22

The presence of signs meeting two or more SIRS criteria is common in all patients in the ICU but is not specific for infection.¹⁵ Among patients in the emergency department, 38% of those with SIRS-positive severe sepsis have an infection,²⁴ as compared with 21% of those with SIRS-negative severe sepsis.²⁴ In our patients with SIRS-positive severe sepsis, the most frequent signs meeting SIRS criteria were an increased heart rate and increased respiratory rate, which is in agreement with the results of a prospective study of SIRS prevalence among patients with infection in the ICU.¹⁵ These observations support the external validity of our study. Our study, however, assessed the prevalence of signs meeting individual SIRS criteria. Among our patients who had infection and organ failure, signs meeting two or more SIRS criteria occurred in 87.9% of patients within 24 hours after ICU admission, supporting their presence in the majority of patients with severe sepsis. Regardless of how we assess their sensitivity, these criteria fail to identify one in eight patients with severe sepsis. Moreover, we found that the cutoff point of two SIRS criteria does not define any specific transition point for risk.

The presence of symptoms meeting two or more SIRS criteria is believed to have excellent sensitivity but low specificity for severe sepsis.¹⁵ Our study showed that it may also have limited sensitivity. The application of the criteria in the first 24 hours after ICU admission (the period when recruitment into sepsis trials is most common) would exclude approximately one in eight ICU patients with infection and organ failure. However, it may also decrease the specificity of these criteria. These patients have substantial mortality and their epidemiologic data are identical to these of patients with classic SIRS-positive severe sepsis,²⁵ which suggests that these two groups represent different clinical phenotypes of the same process. Moreover, on adjusted analysis, mortality increased linearly with each additional SIRS criterion from 0 to 4, with no transitional increase in risk at two criteria to justify the use of two criteria as definitional cutoff point.

Our study has several strengths. It investigates the effect of SIRS criteria within the first 24 hours after ICU admission on the diagnosis of severe sepsis over a period of 14 years. Second, the study is large. Third, the SIRS data consist of physiological or laboratory measurements that were prospectively collected for routine quality-surveillance purposes and are therefore unlikely to be biased. Fourth, our findings are broadly consistent with the limited existing literature and include data from 172 ICUs, which increases external validity, and data obtained in 2013, which increases contemporary relevance.

Our study also has some limitations. The data were collected primarily for quality-control purposes and not for study purposes. We could study symptoms meeting SIRS criteria only during the first 24 hours in the ICU as recorded either every 30 minutes or every 60 minutes on the observation charts. Thus, we cannot comment on their absence or presence before or after this period or for short intervals between observations. We cannot define a population at risk for sepsis to assess the SIRS criteria for their diagnostic value. However, this was not the aim of our study. The accuracy of specific diagnostic coding of infections was not independently monitored. However, on-site audits of data quality, validation rules built into data-collection software, and regular education sessions make systematic bias in diagnostic coding unlikely. Some of the patients who had severe sepsis after surgery may have had SIRS because of the surgery rather than because of sepsis. However, this problem would apply to such patients in any clinical context. Our definition of organ failure related only to the first 24 hours. Thus, patients who might have had organ failure on the second day of the ICU stay were missed. Our definition of organ failure was based on the SOFA score. Other definitions exist,²⁶ which we did not have sufficient data to assess. Our definition of cardiovascular failure relied on the presence of hypotension and a diagnosis of shock and did not include the use of vasopressor drugs. However, such criteria were applied equally to both groups, which makes bias unlikely.

In this epidemiologic study, the requirement of two or more SIRS criteria for the diagnosis of severe sepsis excluded a sizable group of patients in the ICU with infection and organ failure. These patients with SIRS-negative severe sepsis had substantial mortality and, over a period of more than a decade, had epidemiologic characteristics and changes that were essentially identical to those of patients with SIRS-positive severe sepsis, providing indirect empirical evidence that these two groups of patients represent separate phenotypes of the same condition. Moreover, the risk of death in the two groups increased linearly with each additional SIRS criterion from 0 to 4, without a transitional increase in risk at two criteria that would justify this consensus cutoff point. Our findings challenge the sensitivity, face validity, and construct validity of the rule regarding two or more SIRS criteria in diagnosing or defining severe sepsis in patients in the ICU.