A simple scoring system for the prediction of early pregnancy loss developed by following 13,977 infertile patients after in vitro fertilization

Abstract

A retrospective study was conducted to investigate a convenient simple scoring system for the prediction of early pregnancy loss (EPL) based on simple demographics. A total of 13,977 women undergoing transvaginal ultrasound scans on Days 27–29 after in vitro fertilization-embryo transfer (IVF-ET) from June 2016 and December 2017 were included. The first trimester pregnancy outcome was recorded at 12 weeks of gestation. The areas under the curve of this scoring system were 0.884 (95% confidence interval (CI) 0.870–0.899) and 0.890 (95% CI 0.878–0.903) in the training set and test set, respectively. The score totals ranged from -8 to 14 points. A score of 5 points, which offered the highest predictive accuracy (94.01%) and corresponded to a 30% miscarriage risk, was chosen as the cutoff value, with a sensitivity of 62.84%, specificity of 98.79%, positive predictive value (PPV) of 88.87% and negative predictive value (NPV) of 94.54% for the prediction of EPL in the training set. In the test set, a score of 5 points had a sensitivity of 64.69%, specificity of 98.78%, PPV of 89.87% and NPV of 93.62%, and 93.91% of the cases were correctly predicted. Therefore, the simple scoring system using conventionally collected data can be conveniently used to predict EPL after ET. However, considering the limitations, its predictive value needs to be further verified in future clinical practice.

Introduction

The first routine ultrasound scan is commonly performed on Day 28 after in vitro fertilization-embryo transfer (IVF-ET) in most reproductive centers in China to determine the location and viability of the embryo. Although the appearance of cardiac activity at first scan means a higher likelihood of continuing pregnancy, the rate of subsequent miscarriage is between 2 and 16% [1], and there are some examples of pregnancies that were initially thought to have poor viability but eventually developed normally [2]. The mental and psychological pressure on women undergoing IVF-ET is significant, especially when the viability of a pregnancy is uncertain. Under these circumstances, the physician has an important role in rapidly and accurately predicting the pregnancy outcome in an evidence-based and professional manner, as this would be helpful for further consultations and in determining subsequent management.

Individual maternal factors, such as a high maternal age (MA) [3], and abnormal ultrasound parameters, such as embryonic bradycardia and excessively large or small yolk sac diameter (YSD), have been shown to be associated with pregnancy failures in previous studies [4, 5]. Some previous mathematical models involve combinations of individual risk factors and exhibit reasonable performance [2, 6,7,8,9]. However, these models also have some shortcomings, such as an insufficient sample size in most studies, study groups being mixed (natural and assisted pregnancies), inconsistent gestational age (GA), and studies not providing specific probabilities of ongoing pregnancy or miscarriage.

In addition, many patients will return to the local hospital after pregnancy. We aimed to build a model by using ultrasound data obtained from real-time measurements and simple clinical indicators such as MA to quickly predict and assess pregnancy in different hospitals without knowing many of the patient’s assisted fertility indicators.

Therefore, the aim of this study was to investigate a convenient and accurate simple scoring system for the prediction of early pregnancy loss (EPL) based on simple demographics and the first routine ultrasound scan performed on days 27–29 after ET.

Methods

Participants

This retrospective study was conducted at the Reproductive and Genetic Hospital of CITIC-Xiangya (Changsha, Hunan, People's Republic of China). The institutional review board approved this study (date of approval: 26 July 2019; reference number: LL-SC-2019-015; Changsha, China).

The infertile patients included in this study underwent IVF treatment between June 2016 and December 2017. One to 2 embryos with good quality were transferred at Day 3 or Day 5. The embryo morphology was scored according to the criteria by Hardarson et al. [10]. The first routine transvaginal sonography (TVS) scan using a 5–9 MHZ probe (GE VOLUSON 730 or E8, General Electric) was arranged on days 27–29 to observe the number of embryos and their locations and viability. Only intrauterine singleton pregnancies were included. Measurements were taken in accordance with the ISUOG practice guidelines [11] and conformed to uniform standards: gestational sac diameter (GSD) was calculated as the mean value of 3 perpendicular diameters with the calipers placed at the inner edges of the trophoblast; YSD was calculated as the average of 3 perpendicular diameters with the calipers placed at the center of the yolk sac (YS) wall; embryonic length (EL) was measured as the greatest length of the embryo in the anterior to posterior dimension; and embryonic heart rate (EHR) was calculated from frozen M-mode images with electronic calipers by measuring the distance between two heart waves. The presence of intrauterine hematoma (IUH), a hypoechoic or anechoic crescent-shaped area between the chorionic membrane and the myometrium, was also noted. Data on clinical characteristics including day-14 (blastocysts on day 12), serum β-human chorionic gonadotropin (HCG) levels, MA, duration of infertility, infertility type and endometrial (EM) thickness on transfer day were also collected. The first-trimester pregnancy outcomes of these participants were noted at 12 weeks of gestation. Women with a continuing pregnancy for > 12 weeks of gestation were classified as ongoing pregnancy, and women with spontaneous miscarriage before or at 12 weeks of gestation were classified as EPL.

Statistical analysis

Categorical variables are described using frequencies, and continuous variables are described using means and standard deviations. Pearson’s chi-squared test, Fisher’s exact test, T-test or the Wilcoxon rank sum test was used to compare categorical or continuous variables between women with ongoing pregnancies and women with EPL. The cases from the first year (year 2016) were used to generate the training set, and the cases from the second year (year 2017) were used as the verification set. The binary logistic regression (LR) model was used in the training set to identify the probable predictive factors of EPL. Points associated with each category of each risk factor were computed, and the risks associated with point totals were determined according to the “The Framingham Study risk score system” [12]. Based on the area under the curve (AUC) and the clinical value, we evaluated the cutoff value of the prediction model that had a relatively high risk of miscarriage and high prediction accuracy. Then, we used the verification set to evaluate and verify the simple scoring system. All statistical analyses were performed using SPSS version 24.0 software (SPSS, Inc.). The results of each test were considered significant when the two-sided P value did not exceed 0.05, except where otherwise specified.

Results

From June 2016 to December 2017, 23,929 infertile patients conceived clinical pregnancies via IVF-ET in our hospital. A total of 14,118 women with an intrauterine singleton pregnancy were identified during this period and among them, 141 patients were lost to follow-up. Finally, a cohort of 13,977 women were enrolled, including 12,051 patients with ongoing pregnancies and 1926 patients with EPLs.

Comparisons of women with ongoing pregnancies and EPLs

Compared with women with ongoing pregnancies, women with EPLs had significantly higher MA, body mass index, infertility duration and transfer cycle and significantly lower Day 14 hCG and EM thickness on transfer day (p < 0.001). The infertility type, cause of infertility, and insemination methods were also significantly different between women with ongoing pregnancies and women with EPLs (p < 0.05). The number of embryos transferred was not significantly different (p = 0.44).

Based on the TVS measurements, the GSD (18.5 ± 3.6 vs. 13.2 ± 4.8 mm), EL (3.5 ± 0.9 vs. 1.2 ± 1.6 mm), YSD (3.6 ± 0.4 vs. 2.6 ± 1.5 mm) and EHR (114.5 ± 2.2 vs. 42.4 ± 53.5 bpm) were significantly greater in women with ongoing pregnancies than those with EPLs (p < 0.001). The incidence of IUH (16.0% vs. 18.8%, P = 0.002) was also markedly higher in women with EPLs (Table 1).

Table 1 Comparisons of the parameters between 2 groups

Binary LR analysis

After stepwise screening, MA (p = 0.0001, OR 1.096, 95% confidence interval (CI) 1.073–1.119), GSD (p = 0.0001, OR 0.892, 95% CI 0.864–0.921), EL (p = 0.0020, OR 0.783, 95% CI 0.672–0.913), YSD (p = 0.0600, OR 0.853, 95% CI 0.723–1.007), EHR (p = 0.0001, OR 0.966, 95% CI 0.961–0.971) and EM on transfer day (p = 0.0030, OR 0.929, 95% CI 0.884–0.976) were found to be predictive factors of EPL and finally entered the scoring system (Fig. 1).

Fig. 1

Predictive factors for EPL. EPL early pregnancy loss; OR odds ratio; CI confidence interval; MA maternal age (years); EM endometrium (mm); GSD gestational sac diameter (mm); EL embryonic length (mm); EHR embryonic heart rate (bpm); YSD yolk sac diameter (mm)

The scoring system

The points associated with each of the categories of the risk factors were calculated and are shown in Table 2. Table 3 shows the scoring system derived from the binary LR model and the predictive value of each total point. The AUC of this scoring system was 0.884 (95% CI 0.870–0.899) in the training set. The score totals ranged from -8 to 14 points. Scores of 5 and 6 points offered the highest predictive accuracy (94.01% and 93.95%, respectively), and the predicted risks of miscarriage were 30.03% and 39.55%, respectively. Considering that women undergoing IVF were anxious, we chose a score of 5 points corresponding to 30.03% risk of miscarriage as the cutoff value, with a sensitivity of 62.84%, specificity of 98.79%, positive predictive value (PPV) of 88.87% and negative predictive value (NPV) of 94.54%, for the prediction of EPL in the training set.

Table 2 Points associated with each of the risk factor categories

Table 3 Scoring system derived from the binary LR model

In the verification set, the AUC of the scoring system was 0.890 (95% CI 0.878–0.903). A score of 5 points had the highest diagnostic accuracy, with 93.91% of the samples correctly predicted. The sensitivity, specificity, PPV and NPV were 64.69%, 98.78%, 89.87% and 93.62%, respectively (Table 4, Additional file 1: Table S1).

Table 4 The classification of the results obtained using a score of 5 as the cutoff value

Discussion

In this study, we collected the demographic and ultrasound findings for days 27–29 after ET and constructed a simple scoring system for the prediction of EPL, with AUCs of 0.884 and 0.878 in the training set and verification set, respectively. Point 5 had the highest predictive accuracy and was recommended as the cutoff value for clinical practice.

The process of achieving pregnancy through IVF is usually very difficult for an infertile woman. Thus, when the woman becomes pregnant by IVF, she is usually very anxious about the development of the embryo, even after the detection of cardiac activity but especially if an empty gestational sac or only a YS are detected. Generally, for patients who have well-developed embryos at the first routine TVS examination on days 27–29 after ET, the next ultrasound scan will usually be scheduled on day 45 after ET; however, for patients with embryos of uncertain viability, the recommendation is usually to have another TVS scan 7–10 days after the first scan to assess the development and viability of the embryo.

If the pregnancy outcome can be predicted in advance, the anxiety of the pregnant woman can be greatly relieved. Many models have been constructed to predict pregnancy outcomes effectively [13,14,15]. Among them, LR analysis has been the most commonly used [15, 16]. The overall goal of this study was to construct a simple and practical scoring system that is similar to the “Apgar score system” using simple demographic and ultrasound findings on days 27–29 after ET, as this would be easy for clinical application, especially for patients without sufficient assisted reproduction indicators when they return home after pregnancy. Through this system, we can inform patients with a specific probability of miscarriage after the first routine TVS examination, which may help reduce the patient’s anxiety and psychological burden and provide guidance for follow-up decisions.

The findings of this study are consistent with the current knowledge that miscarriage is more likely with increasing MA [17], low hCG level [18], low EM thickness on transfer day [19] and the presence of IUH [20] and that miscarriage is less likely after the visualization of embryonic cardiac activity [15]. Accordingly, in our scoring system, a greater MA was associated with a higher score, while greater GSD, YSD, EL and EHR were all associated with lower scores. However, both small and large YSDs corresponded to higher scores. A higher score indicates a greater contribution to miscarriage. Thus, the possibility of miscarriage increased with the total score. A score of 5 offered the highest predictive accuracy in both the training set and verification set, and the corresponding miscarriage risk exceeded 30%, which was not a low risk for anxious IVF patients. Therefore, we recommend using a score of 5 as a clinical threshold for warning patients of this risk and providing more counselling about miscarriage. In practical applications, we hope that the false-positive rate (FPR) will be as low as possible, as this may lead to unnecessary medical treatment. A low FPR requires high specificity. In this system, the specificity of scores -8 to 0 was not satisfactory. Thus, this is a scoring system for determining the probability of miscarriage that could allow doctors and patients to know the risks in advance, but because miscarriage cannot be prevented, the outcome cannot be changed by this scoring system. The advantage of this scoring system is that it is easily transferable to clinical use, where both maternal and ultrasound variables are easily available and the calculation is simple. For example, in a woman with a MA of 38 years and who has an EM of 8 mm on transfer day, a GSD of 10 mm, a YSD of 2.2 mm, an EL of 2.5 mm and an EHR of 88 bpm by TVS on Day 28 after ET, the miscarriage score is 9, and the estimated risk of miscarriage is 70.67% (Fig. 2). Even though the results indicate a strong likelihood of EPL, there is still a significant risk that this is a false-positive finding. Thus, it is imperative to repeat the ultrasound scan for this patient 7–10 days later to confirm the viability of the embryo before performing any medical interventions. In contrast, if a patient does not present with any symptoms and has a low total score (for example, 0), no follow-up TVS scan is needed until Day 45 after ET.

Fig. 2

Risk of EPL in infertile women with an intrauterine singleton pregnancy based on the scoring system. EPL early pregnancy loss; MA maternal age (years); EM endometrium (mm); GSD gestational sac diameter (mm); EL embryonic length (mm); EHR embryonic heart rate (bpm); YSD yolk sac diameter (mm)

A previous study [21] constructed a similar scoring system to predict pregnancy viability and achieved an AUC of more than 0.90. However, this study focused on natural conceptions, and the ultrasound variables and clinical characteristics collected in that study were collected within a time period of GA < 84 days. The measurement methods may also have differed from ours. However, in our study, all ultrasound parameters were collected at the same time on Days 27–29 after ET, and the specific EHR value was included, not just the presence or absence of embryonic cardiac activity. IUH was unexpectedly not included in the final system. We speculate that this may be because IUH occurs more frequently in pregnancies after IVF than in spontaneous conceptions [22], not only in women with EPLs but also with ongoing pregnancies.

We note that the sensitivity of this simple model was not ideal, which might indicate that for the IVF population, including only ultrasonic measurements and simple clinical indicators had limited predictive efficacy, and in further studies, relevant indicators for assisted pregnancy should be added to improve the predictive efficacy. A major limitation was that since there was no information on bleeding, abdominal pain or smoking history in our hospital's electronic medical record system during the study period, these indicators were not included in our system, but they might further improve the predictive performance if included. Since this system was derived from the IVF population, its application in the general population has yet to be validated.

In conclusion, we have developed a simple and practical scoring system that provides a probability for EPL based on the simple demographic and ultrasound findings obtained on days 27–29 after ET. This system is easy and simple for clinical use. Point 5 is recommended as the clinical threshold for warning patients of an EPL risk. When the predictive result is a high risk of EPL, repeated scans are recommended 7–10 days later to confirm the viability of the embryo. When the predictive result is a low risk of EPL and patients have no symptoms, the next examination can be performed on day 45 after ET.