Reliability of point-of-care ultrasound to evaluate fluid tolerance performed by critical care residents

Abstract

Background

Patients with hypotension usually receive intravenous fluids, but only 50% will respond to fluid administration. We aimed to assess the intra and interobserver agreement to evaluate fluid tolerance through diverse ultrasonographic methods.

Methods

We prospectively included critically ill patients on mechanical ventilation. One trained intensivist and two intensive care residents obtained the left ventricular outflow tract velocity–time integral (VTI) variability, inferior vena cava (IVC) distensibility index, internal jugular vein (IJV) distensibility index, and each component of the venous excess ultrasound (VExUS) system. We obtained the intraclass correlation coefficient (ICC) and Gwet’s first-order agreement coefficient (AC1), as appropriate.

Results

We included 32 patients. In-training observers were unable to assess the VTI-variability in two patients. The interobserver agreement was moderate to evaluate the IJV-distensibility index (AC1 0.54, CI 95% 0.29–0.80), fair to evaluate VTI-variability (AC1 0.39, CI 95% 0.12–0.66), and absent to evaluate the IVC-distensibility index (AC1 0.19, CI 95% − 0.07 to 0.44). To classify patients according to their VExUS grade, the intraobserver agreement was good, and the interobserver agreement was moderate (AC1 0.52, CI 95% 0.34–0.69).

Conclusions

Point-of-care ultrasound is frequently used to support decision-making in fluid management. However, we observed that the VTI variability and IVC-distensibility index might require further training of the ultrasound operators to be clinically useful. Our findings suggest that the IJV-distensibility index and the VExUS system have acceptable reproducibility among in-training observers.

Background

Patients with hypotension usually receive intravenous fluids to increase preload. However, only 50% will be fluid responders by increasing their stroke volume (SV) and cardiac output (CO) [1]. Moreover, excessive fluid administration is associated with mortality [2]. Therefore, determining fluid tolerance, which is the moment a patient can tolerate the administration of fluids without causation of organ dysfunction, helps to avoid excessive fluid administration [3].

Positive-pressure ventilation induces cycling changes in the load of the left and right ventricles. These cyclic changes are greater in fluid responders. Thus, SV variability is a good predictor of fluid responsiveness [4].

A transthoracic echocardiogram allows us to obtain the cross-sectional area (CSA) of the left ventricular outflow tract (LVOT) and the velocity–time integral (VTI) to estimate the SV (SV = CSA × VTI) [5]. Since the CSA is constant during the cardiac cycle, VTI variability also predicts fluid responsiveness [5, 6]. When performed by experienced echocardiographers, the VTI has a good interrater agreement [7]. However, when VTI is measured by medical students after a short training session, its reproducibility decreases [8].

Likewise, ultrasound allows for evaluating inferior vena cava (IVC) diameter changes during the respiratory cycle to predict fluid responsiveness. The IVC distensibility index (100 × [maximum diameter-minimum diameter/minimum diameter]) is widely used. However, its ability to predict fluid responsiveness varies across studies [9, 10]. Most studies evaluating the reproducibility of IVC measurements have been performed in the emergency department and have shown variable interrater agreement [11, 12].

The internal jugular vein (IJV) distensibility index has been evaluated as a predictor of fluid responsiveness [13, 14]. However, there are no studies evaluating the IJV distensibility index interrater agreement.

In addition to cardiac output, organ perfusion depends on the arteriovenous pressure gradient [15]. Therefore, venous congestion due to fluid overload may impair organ perfusion. To evaluate venous congestion through ultrasound, the recently developed Venous Excess Ultrasound (VExUS) score incorporates the IVC diameter and venous Doppler waveform of the portal, hepatic, and interlobular renal veins. The VExUS score has been shown to predict acute kidney injury following cardiac surgery [16]. Currently, there are no studies that evaluated interrater agreement in critically ill patients.

Methods

Aim, design, and setting

This study aimed to assess the reliability of point-of-care ultrasound to evaluate fluid tolerance through diverse methods performed by observers with different levels of training. This prospective study was performed in an academic cancer center.

Participants

We included consecutive patients admitted to the intensive care unit (ICU) from November 2020 to July 2021. Eligible patients were above 18 years old and required mechanical ventilation. Patients with atrial fibrillation or mechanical ventilation in a prone position were excluded.

Procedures

After inclusion, we registered demographic data, comorbidities, body mass index (BMI), Sequential Organ Failure Assessment (SOFA) score, Richmond Agitation–Sedation Scale, and vital signs. Ultrasonographic evaluations were performed with SonoSite M-Turbo equipment.

All ultrasonographic assessments were performed by three observers. Each observer performed the first assessment. The observers were blinded to each other’s results.

To assess the intraobserver agreement, once the three observers have finished, they performed a second assessment. This measure could have a potential recall bias that must be considered in the interpretation of the results. However, the short time between the first and the second observation prevents the influence of changes due to the clinical condition of critically ill patients.

Observer 1 was a graduate intensivist with postgraduate training in critical care ultrasound and five years of experience. Observers 2 and 3 were intensive care residents with at least three months of training. The current residence program comprises theoretical–practical training for four weeks, followed by a period in which they perform at least thirty ultrasonographic volume assessments under supervision by a graduate intensivist for 2 to 3 weeks. Conventionally, after this time it is considered they can incorporate ultrasonographic fluid assessment into their clinical practice as residents.

During the protocol assessment, patients were placed in a supine position with the head at 30°, mechanical ventilation in mandatory mode, with plateau pressure less than 30 cmH2O, peripheral oxygen saturation (SpO2) more than 95%, and end-expiratory pressure (PEEP) less than 10 cmH2O.

Left ventricular outflow tract velocity–time integral assessment

The LVOT VTI was evaluated with a sectorial transducer. The probe was placed in the apical position, in the five-chamber view, applying pulsed-wave Doppler at the level of the LVOT. Maximum and minimum values were obtained over five cardiac cycles. LVOT VTI variability was calculated as follows: 100 × [(maximum VTI − minimum VTI)/ (maximum VTI + minimum VTI)/2]. A value of 15% or more was considered a predictor of fluid responsiveness [7].

Inferior vena cava assessment

The IVC diameter was obtained with a 5-MHz convex transducer. The probe was placed in the subxiphoid position to obtain a long-axis view. The IVC diameter was evaluated during a respiratory cycle. The IVC distensibility index was estimated with the formula [(maximum diameter − minimum diameter)/minimum diameter] × 100. An IVC distensibility index equal to or greater than 18% was considered a predictor of fluid responsiveness [9].

Jugular vein assessment

The IJV was evaluated with a 10-MHz linear transducer. The IJV was located with two-dimensional ultrasound at the level of the cricoid cartilage. The IJV was identified by compression and color Doppler. We measured the anteroposterior diameter with M-mode ultrasound during a respiratory cycle. The IJV distensibility index was estimated with the formula [(maximum diameter − minimum diameter)/minimum diameter] × 100. An IJV distensibility index equal to or greater than 18% was considered a predictor of fluid responsiveness [13].

Venous excess ultrasound grading system assessment

The components of the VExUS protocol were evaluated with a 5-MHz convex transducer. To obtain the IVC long-axis view, we placed the probe in the subxiphoid position and obtained the IVC maximum diameter. The original protocol indicates that only in the case of an IVC diameter equal to or greater than 2 cm, the hepatic, portal, and renal veins should be evaluated. However, our study aimed to evaluate the reliability of the measurements; therefore, we performed a complete evaluation independently of the IVC diameter.

We obtained a two-dimensional image of the hepatic veins and applied color flow Doppler and pulse-wave Doppler to assess the diastolic and systolic waveforms. Then, we placed the probe in the right midaxillary line to obtain a two-dimensional image of the right portal vein and initiated pulse-wave Doppler. Then we calculated the pulsatility index with the formula: (maximum velocity-minimum velocity)/maximum velocity. We placed the probe in the posterior axillary line, applied color flow Doppler to find the intrarenal lobar vessels, and used pulse-wave Doppler to evaluate the waveform.

Observers evaluated and classified IVC into two categories according to its diameter: < 2 cm and ≥ 2 cm; hepatic vein in three categories: normal (systolic wave greater than diastolic), mild hepatic vein abnormality (systolic wave smaller than diastolic), and severe hepatic vein abnormality (systolic wave reversal); portal vein in three categories: normal (< 30% pulsatility index), mild portal vein abnormality (30–49% pulsatility index), and severe portal vein abnormality (> 50% pulsatility index); and renal vein in three categories: normal (continuous monophasic flow), mild intrarenal vein abnormality (discontinuous biphasic flow), and severe intrarenal vein abnormality (discontinuous monophasic flow). Observers also assigned a VExUS score as follows: Grade 0 (no congestion): IVC < 2 cm, Grade 1 (mild congestion): IVC ≥ 2 cm and any combination of normal or mildly abnormal patterns, Grade 2 (moderate congestion): IVC ≥ 2 cm and one severely abnormal pattern, Grade 3 (severe congestion) IVC ≥ 2 cm and two or more severely abnormal patterns [15, 16].

We evaluated the agreement to categorize each VEXUS component and to obtain the total VEXUS score.

Statistical analysis

Considering a Type I error of 0.05, a Type II error of 0.20, and an expected correlation coefficient of 0.6 among observers, we estimated a minimum sample size of 19 patients. Numerical variables are shown as the median and interquartile range (IQR), and categorical variables are shown as proportions (%). We obtained the intraclass correlation coefficient for numerical variables and Gwet’s first-order agreement coefficient (AC1) with the percentage of agreement for categorical variables. We used Altman´s scale to evaluate the strength of agreement [17]. Analyses were processed by STATA 14.0.

Results

We included 32 patients; their main characteristics are shown in Table 1.

Table 1 Clinical characteristics

The most experienced observer obtained all of the ultrasonographic measurements, but the in-training observers were unable to assess the LVOT VTI for two patients. The remaining ultrasonographic parameters were obtained in all cases by the three observers (Table 2).

Table 2 Obtained values during the first evaluation

Considering the obtained values during the first assessment performed by the three observers, patients were classified as potentially fluid responders in 66.3% (61/92) of VTI evaluations, 76% (73/96) of IJV evaluations, and 36.5% (35/96) of IVC evaluations, and this difference among predictors was statistically significant (p < 0.001). According to the VExUS system, observers reported the absence of venous congestion in 53% (51/96) of the evaluations.

Observers showed poor agreement in obtaining the same numerical value of the VTI-variability, IVC-distensibility index, and IJV-distensibility index. However, when these measurements were categorized, according to their cut-off points to predict fluid responsiveness, we observed better agreement among the observers (Table 3).

Table 3 Intraobserver and interobserver agreement

The most experienced observer obtained moderate intraobserver agreement to classify patients as fluid responders with IVC, and IJV. However, in-training observers obtained variable intraobserver agreement for the different methods.

To classify patients as fluid responders, the obtained interobserver agreement was moderate to evaluate the IJV-distensibility index, fair for VTI variability, and absent for IVC variability.

To evaluate venous congestion through the VExUS system, observers obtained a good intraobserver agreement and a moderate interobserver agreement. Despite in-training observers showing a fair intraobserver agreement to evaluate the portal pattern, all observers obtained a good to very good intraobserver agreement assessing the remaining components of the VExUS system.

Discussion

We evaluated the agreement to acquire and interpret ultrasonographic images performed by one intensivist and two critical care residents. Only the most experienced observer was able to achieve the LVOT VTI assessment in all patients. Villavicencio et al. previously reported an inability to obtain adequate LVOT images in half of the patients evaluated by intensivists with only basic critical care echocardiography training [18]. Previous studies have shown that LVOT VTI assessment relies on the observer experience [7, 8]. However, is a useful tool to predict volume response and to estimate SV [5]. Our findings suggest that extensive and focused training is required before incorporating LVOT VTI into clinical assessment.

Although the IVC distensibility index is a widely used method, we observed low reproducibility among in-training observers. Most studies assessing the agreement between experienced and in-training observers to evaluate IVC-distensibility were performed in the emergency department; they included spontaneously breathing subjects and reported poor to moderate interrater agreement [19, 11]. Rollas et al. reported moderate agreement among intensive care specialists and residents, but only seven patients were mechanically ventilated, and none had a BMI over 30 kg/m² [20]. Our lower agreement may be because all of our patients were mechanically ventilated, and 25% had a BMI over 30 kg/m². These variables could alter the accuracy of the IVC measurements [21].

Recently, the IJV has been proposed as a useful tool to predict fluid responsiveness compared to invasive methods [13]. Additionally, a correlation between the IJV measurement and the central venous pressure in spontaneously breathing patients has been demonstrated [22]. However, its correlation with IVC measurement is not clear [14, 22]. To our knowledge, this is the first study evaluating IJV-distensibility agreement among experienced and in-training observers, and we found moderate agreement, which suggests that it could be a potentially useful tool for in-training physicians.

Currently, the evaluation of venous congestion has become more relevant in the clinical setting. The VExUS system is a novel, noninvasive method to assess venous congestion. Interrater agreement to obtain and interpret the VExUS system has not been previously evaluated. Our results suggest that it could be a reproducible method, even among less experienced observers. Even though intrarenal vein assessment could be difficult because of their small size, we observed a good to very good intraobserver agreement and a moderate interobserver agreement. Previous studies have shown absolute intra and interobserver agreement for classifying the three renal vein patterns that we used [23, 24].

Our results support the need to analyze the agreement among in-training observers to identify the most difficult procedures, reinforce their teaching, and ensure the accuracy of the methods before their clinical use.

This is a single-center study, and we only evaluated three observers (one graduate intensivist and two intensive care residents), which limits the generalizability of the results. Despite being beyond the scope of this study, it is worth mentioning that we did not use a gold standard to define fluid responsiveness, which would have made it possible to establish a comparison among the different predictors of fluid responsiveness.

Conclusions

Point-of-care ultrasound is frequently used to support decision-making in fluid management. However, we observed that the VTI variability and IVC distensibility index might require further training of the ultrasound operator to be clinically useful. Our findings suggest that the IJV-distensibility index and the VExUS system have acceptable reproducibility among in-training observers.