Translation and validation of the STOP-Bang questionnaire into Slovene
European Journal of Medical Research volume 26, Article number: 32 (2021)
To translate, culturally adapt and evaluate the Slovene version of the STOP-Bang questionnaire (SBQ) for use in the sleep clinic.
Standard forward–backward translation and harmonisation of the Slovene translation of the SBQ were performed. Test–retest reliability was performed on a sample of healthy subjects. A cross-sectional study was performed with patients referred for a sleep study. Patients filled out the Slovene translation of the SBQ before undergoing sleep study.
The validation group consisted of 256 patients, of which 237 (92.6%) were included. Mean age was 52.5 ± 14.6, 63.3% of patients were male. Obstructive sleep apnoea (OSA) (apnoea–hypopnea index (AHI) ≥ 5) was present in 69.6% of patients, of whom 22.4% had mild (AHI ≥ 5 and < 15), 21.9% moderate (AHI ≥ 15 and < 30), and 25.3% severe (AHI ≥ 30) OSA. A SBQ score of 3 had a sensitivity of 92.1 (86.9–95.7), specificity of 44.4 (32.7–56.6), PPV of 79.2 (75.5–82.4) and AUC of 0.757 (95% CI 0.692–0.823; p < 0.001) for all OSA (AHI ≥ 5). Each increase in the SBQ score was associated with an increase in the probability of OSA.
This study shows that the Slovene version of the SBQ is a valid tool for evaluating the risk of OSA in a sleep clinic.
Obstructive sleep apnoea (OSA) is the most common sleep-related respiratory disorder which is recognized as an independent risk factor for a range of clinical conditions, such as hypertension, stroke, depression and diabetes [1, 2]. Moreover, OSA is a significant cause of motor vehicle crashes  and is associated with an increase in all-cause mortality, particularly due to coronary artery disease [3, 4]. It has been estimated that up to 80% of individuals with moderate-to-severe OSA have not been diagnosed .
Polysomnography as the current gold standard for OSA diagnosis is expensive and difficult to set up and interpret . Portable home monitoring (type III polygraphy) was approved by the American Academy of Sleep Medicine as an alternative in patients without significant cardiorespiratory disease, chronic opioid medication use, history of stroke, or severe insomnia .
High prevalence of undiagnosed OSA, limited resources and the short- and long-term consequences of the disease have created a need to develop a reliable and affordable screening tool for OSA risk stratification. Questionnaires can be appropriate tools to that end since they can be applied and scored easily as part of routine daily practice . The STOP-Bang questionnaire (SBQ) is a simple and validated questionnaire that detects OSA with high sensitivity and is, therefore, more suited for a sleep clinic setting compared other questionnaires as it helps to avoid missing cases [8, 9]. In their 2017 meta-analysis, Chiu et al.  compared the Berlin questionnaire, the SBQ and Epworth’s sleepiness scale in terms of OSA detection. The results revealed that for mild, moderate, and severe OSA, the pooled sensitivity and diagnostic odds ratio of the SBQ were significantly higher in comparison to the other screening questionnaires. The SBQ also demonstrated good flexibility as it had the largest area under the curve when compared to seven other questionnaires for the commonly used AHI cutoffs of 5, 15 and 30 . The SBQ has been translated and validated in numerous languages, but no scientifically produced translation or validation of a Slovene version has, thus, far been produced. We aim to translate, culturally adapt and validate the SBQ for use with Slovene patients.
We split our study into two parts: first, we translated and adapted the SBQ and tested its internal consistency; second, the translated SBQ was validated against sleep study in a cross-sectional study.
From February to April 2017, a sample of 153 healthy Slovene-speaking subjects aged 18 or older were recruited for test–retest reliability. 134 (87.6%; mean age 42.9 ± 12.7) completed both sets of the SBQ required for final analysis. The demographic characteristics and SBQ results of this sample are shown in Table 1.
The second part of the study was conducted at the sleep clinic at the Institute of Clinical Neurophysiology, University Medical Centre Ljubljana, Slovenia. All patients referred for polygraphy or polysomnography who were 18 or older and spoke Slovene were asked to participate in the study. Patients with neuromuscular conditions were excluded. There were no limitations of referrals. Of the 256 patients referred, 237 (92.6%; mean age 52.5 ± 14.6) were included in the final analysis. 16 patients failed to complete the questionnaires. Two patients were excluded for low fidelity polygraphy recordings which they were not willing to repeat. One was excluded, as he could not fall asleep with PG and declined further testing. The demographic characteristics of this second sample are shown in Table 2.
Study design and data collection
Step 1: Translation of the SBQ
The SBQ was translated from English to Slovene by two independent researchers, one a medical doctor with experience in sleep medicine and the other a psychologist with experience in instrument development and translation. Both were native Slovene speakers proficient in English. A bilingual panel consisting of the two researchers who performed the forward translation and a medical doctor, a somnology specialist, conducted a synthesis of the two translations. An independent translator, psychologist by training, with no knowledge of the SBQ, who grew up in a Slovene–English bilingual home, conducted the back translation. To verify that the questions were understood correctly 10 adults (6 females and 4 males, mean age 39.3 ± 11.8) participated in a one-on-one think-aloud cognitive interview with a psychologist. Procedures were in line with the standards set out by the World Health Organisation .
Step 2: Test–retest reliability of the SBQ
Participants were given two sets of questionnaires. The first set contained demographic questions, exclusion criteria and the SBQ. The second set consisted of the SBQ retest which was taken 2–3 weeks after the first.
Step 3: Validation
Polysomnography and polygraphy
Patients referred to the sleep clinic for sleep study underwent either ambulatory type III polygraphy (PG) or type I polysomnography (PGS). PSG was used in patients with significant cardiorespiratory disease, chronic opioid medication use, history of stroke, or severe insomnia. When the PG recording could not be used due low fidelity or failure because of technical reasons such as the respiratory effort belt becoming loose, the dislodging of the pulse oximeter, etc. PSG was ultimately performed. This is in line with normal sleep centre operations and follows the recommendations by the American Association for Sleep Medicine (AASM) . PG was recorded using the Alice NightOne, Phillips Respironics, system. PSG was recorded using the Alice 6, Phillips Respironics, system. Patients with neuromuscular conditions were excluded. This was done because other disorders such as sleep-onset insomnia, sleep maintenance insomnia, excessive eye movement sleep behaviour disorder, central sleep apnoeas, and diaphragm weakness with pseudo-central apnoeas, for which the SBQ was not designed, are common in this population .
Recordings were manually scored in our accredited sleep centre by our most experienced certified sleep specialist. Scoring was conducted in accordance with AASM guidelines and rules [6, 14]. The scorer was blinded to patients’ clinical histories and SBQ scores. The severity of OSA was determined as mild for apnoea hypopnea index (AHI) ≥ 5 and < 15, moderate for AHI ≥ 15 and < 30, and severe for AHI ≥ 30 [6, 14]. In the final analysis, datasets with complete STOP-Bang questionnaires and good-quality recordings were included.
Patients’ characteristics were presented with the mean (standard deviation) in the case of normally distributed numerical variables; median (interquartile range) in the case of non-normally distributed numerical variables; and with frequencies (%) in the case of categorical variables. The differences between the OSA and non-OSA group were tested with independent t test or Mann–Whitney test, while the Chi-square test was used for categorical variables. The validation of the SBQ included the evaluation of internal consistency (Cronbach’s Alpha) and test–retest reliability (Gwet’s AC1 agreement coefficient). Reliability was also tested with factor analysis using tetrachoric correlations. To assess the predictive validity of the SBQ, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were calculated for different AHI cutoffs. Logistic regression was used to calculate the predicted probabilities for AHI scores.
Translation and adaptation
After the forward translations were finished, they were examined by a bilingual panel and a consensus translation was reached. No major discrepancies were noted between the original English and the back-translated version. After reviewing the results of cognitive interviewing, the panel decided to break down the question whether the body mass index (BMI) was 35 or higher into its constituents, meaning that patients were instead asked to give their weight and height as this was easier to self-report. See Additional file 1 for the final version of the Slovene SBQ.
Internal consistency and temporal stability
Cronbach’s Alpha coefficient for the 8 items was 0.628. Factor analysis suggested a single-component solution that explained 32.2% of the total item variance. The loadings for each of the SBQ items are presented in Table 3. The intraclass correlation coefficient between test and retest total scores was 0.94 (95% CI 0.91–0.95, p < 0.001). The test–retest reliability for each item was assessed with Gwet’s AC1 coefficients and almost all scores were greater than 0.9, indicating excellent test–retest reliability (Table 4).
Validation against AHI
A comparison of the data for patients who had OSA (AHI ≥ 5) with those who did not reveals a significant difference in demographic characteristics between the two groups (Table 5).
Of the 237 patients at the sleep clinic, 72 patients (30.4%) had no OSA (AHI < 5), whereas the remaining 165 patients had mild (n = 53, 22.4%), moderate (n = 52, 21.9%) and severe OSA (n = 60, 25.3%). See Table 6 for comparison of the answers given to the SBQ and AHI between patients with and without OSA. As shown in Fig. 1, the total SBQ scores were positively associated with the AHI score. The correlation coefficient was 0.56 and statistically significant (95% CI 0.47–0.64; p < 0.001). 5 patients underwent PSG, while the rest had PG. The SBQ was evaluated with the cutoff values of AHI ≥ 5, 15 and 30; the respective areas under the ROC curve were 0.757 (95% CI 0.692–0.823; p < 0.001), 0.768 (95% CI 0.711–0.825; p < 0.001) and 0.77 (95% CI 0.704–0.836; p < 0.001). Plots for age, BMI, sex and neck circumference against AHI are also given for comparison in Figs. 2, 3, 4 and 5, respectively. As the SBQ score increased, sensitivities and NPV decreased whereas specificities and PPV increased. Detailed results are presented in Table 7. Predicted probabilities of having OSA based on the SBQ were calculated. As the SBQ increased from 3 to 8, the probabilities of OSA also increased (Fig. 6).
During the process of cognitive interviewing, it was suggested that the question of BMI be broken down into its components, body mass and height, which would make it easier to use and understand. We chose to adopt this suggestion. We were unable to find any other validation studies of the SBQ where this was done.
Our translation of the STOP-Bang questionnaire showed good temporal stability. Intraclass correlation coefficient between test and retest total scores was high.
Internal consistency was evaluated with two statistical methods. The first was calculating Cronbach’s alpha, which was somewhat low at 0.63. This on par with 0.62 for the Brazilian translation , close to 0.7 for the Arab translation  and higher than the Lithuanian translations’ 0.41 . We also performed factor analysis, which is more suitable for dichotomous variables, [18, 19] such as are found in the SBQ. This showed good internal consistency for six out of the questionnaire’s eight items. Item number two “Do you often feel tired, fatigued or sleepy during the daytime?” and item six “Age older than 50?” had a loading score below the threshold of 0.3. For item two, this could be explained by other common causes of tiredness other than OSA, for instance depression, which are common in the general population. Another cause could be the fact that some patients with OSA do not experience excessive daytime sleepiness . The low factor loading of item five that refers to age could perhaps be explained by the fact that the study population consisted mostly of middle-aged subjects and a larger sample could perhaps have been more telling. Dr. Chung, who designed the questionnaire, did not evaluate internal consistency, citing that the questionnaire reflected four different dimensions of OSA morbidity and that internal consistency checking was, thus, not applicable . Internal consistency checking was nevertheless performed in certain validation studies [7, 16] and omitted in others [17, 22, 23]. When it was carried out, Cronbach’s coefficient alpha was used, and values were typically low.
The prevalence of OSA (AHI of ≥ 5) in our sleep clinic population was 69.6%. For the Portuguese version, the prevalence was 78% , for the Lithuanian, this was 93% , for the Arabic, it was 94%  and for the Malayan, it was 100% .
A comparison of the answers given by patients with and without OSA (AHI ≥ 5) showed significant differences in all but one of the eight questions, i.e. the question referring to a BMI ≥ 35 where the p value was 0.113. This was somewhat surprising considering that OSA has been reported in over 40% of persons with a BMI of more than 30 . Nevertheless, of the 44 patients with a BMI ≥ 35, only 9 did not have OSA. Interestingly, Reis et al.  also found BMI to be statistically nonsignificant. The specific cutoff value used for the BMI might be the cause. This sentiment is supported by Fig. 3 and Table 5 which show a correlation between the BMI and AHI.
For a SBQ score of 3, we found that the area under the ROC curve was high at 0.757 (95% CI 0.692–0.823; p < 0.001) for all OSA (AHI ≥ 5). This increased slightly for moderate/severe and severe OSA to 0.768 (95% CI 0.711–0.825; p < 0.001) and 0.77 (95% CI 0.704–0.836; p < 0.001), respectively. The AUC for all OSA (AHI ≥ 5) obtained by Reis et al.  was slightly higher than ours at 0.806 (95% CI 0.730–0.881), but slightly lower for moderate/severe (AHI ≥ 15) at 0.730 (95% CI 0.661–0.798) and severe OSA at 0.728 (0.655–0.801).
Among patients referred to the sleep clinic the Slovenian version of SBQ, at a score of 3, showed a high sensitivity 92.1% (86.9–95.7%) and moderate specificity of 44.4% (32.7–56.6%) for all OSA (AHI ≥ 5). This was on par with benchmarks such as Chung et al. , who had a sensitivity of 72.1% and specificity of 38.2% and Silva et al.  with sensitivity of 82.0% and specificity of 43.3% for the same range and cutoff. Low specificity was also observed in a number of translations [7, 16, 17] as well as meta-analysis by Nagappa et al. .
The PPV for an SBQ score of 3 for any OSA (AHI ≥ 5) was high at 79.2 (75.5–82.4). Specificity and PPV increased continuously for every increase in the SBQ. These results were on par with other translations of the SBQ [7, 24]. High sensitivity and PPV are essential for screening tools, but it could be argued that NPV is perhaps even more important for risk stratification. Our results show that a STOP-Bang score of 2 had a NPV of 80.0% (46.6–94.8) for all OSA (AHI ≥ 5) and 100.0% for moderate/severe (AHI ≥ 15) and severe OSA (AHI ≥ 30). Although our NPV might be higher due to an underestimation of the AHI brought about by the high percentage of sleep studies conducted with PG, the results are similar those obtained by Portuguese researchers .
An important study limitation was that the population referred to the sleep clinic was in a sense already pre-screened by referring practitioners. Our findings, thus, cannot be extended to other settings.
In our study, we primarily utilized PG, which accounted for 97.8% of all recordings. PG devices do not include sleep staging and can give lower AHI compared with PSG where periods of wakefulness are excluded from the calculation of AHI . PG has, however, been shown to be a reliable alternative to PSG and is becoming ever more prevalent in clinical practice [7, 26]. Ours was not the first study to have used PG for validation of the STOP-Bang questionnaire .
Our study has shown that the Slovene version of the SBQ is a simple, reliable and valid tool for the stratification OSA risk among Slovenes referred to a sleep clinic with high sensitivity and moderate specificity.
Availability of data and materials
Data and material are available upon request.
Sleep scoring was done using AASM visual scoring guidelines Sleepware G3 version 3.9.1 by Respironics inc. PA, USA. Statistical calculations and statistical graphic representations were produced using R Core Team (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria.
Eastwood PR, Malhotra A, Palmer LJ, Kezirian EJ, Horner RL, Ip MS, Thurnheer R, Antic NA, Hillman DR. Obstructive sleep apnoea: from pathogenesis to treatment: current controversies and future directions. Respirology. 2010;15(4):587–95. https://doi.org/10.1111/j.1440-1843.2009.01699.x.
Neves Junior JAS, Fernandes APA, Tardelli MA, Yamashita AM, Moura S, Tufik S, Silva H. Cutoff points in STOP-Bang questionnaire for obstructive sleep apnea. Arq Neuropsiquiatr. 2020;78(9):561–9. https://doi.org/10.1590/0004-282x20200086.
Kendzerska T, Gershon AS, Hawker G, Leung RS, Tomlinson G. Obstructive sleep apnea and risk of cardiovascular events and all-cause mortality: a decade-long historical cohort study. PLoS Med. 2014;11(2):e1001599. https://doi.org/10.1371/journal.pmed.1001599.
Punjabi NM, Caffo BS, Goodwin JL, Gottlieb DJ, Newman AB, O’Connor GT, Rapoport DM, Redline S, Resnick HE, Robbins JA, Shahar E, Unruh ML, Samet JM. Sleep-disordered breathing and mortality: a prospective cohort study. PLoS Med. 2009;6(8):e1000132. https://doi.org/10.1371/journal.pmed.1000132.
Young T, Evans L, Finn L, Palta M. Estimation of the clinically diagnosed proportion of sleep apnea syndrome in middle-aged men and women. Sleep. 1997;20(9):705–6.
Kapur VK, Auckley DH, Chowdhuri S, Kuhlmann DC, Mehra R, Ramar K, Harrod CG. Clinical practice guideline for diagnostic testing for adult obstructive sleep apnea: an American academy of sleep medicine clinical practice guideline. J Clin Sleep Med. 2017;13(3):479–504. https://doi.org/10.5664/jcsm.6506.
Reis R, Teixeira F, Martins V, Sousa L, Batata L, Santos C, Moutinho J. Validation of a Portuguese version of the STOP-Bang questionnaire as a screening tool for obstructive sleep apnea: analysis in a sleep clinic. Rev Port Pneumol. 2015;21(2):61–8. https://doi.org/10.1016/j.rppnen.2014.04.009.
Silva GE, Vana KD, Goodwin JL, Sherrill DL, Quan SF. Identification of patients with sleep disordered breathing: comparing the four-variable screening tool, STOP, STOP-Bang, and Epworth sleepiness scales. J Clin Sleep Med. 2011;7(5):467–72. https://doi.org/10.5664/jcsm.1308.
Nagappa M, Liao P, Wong J, Auckley D, Ramachandran SK, Memtsoudis S, Mokhlesi B, Chung F. Validation of the STOP-Bang questionnaire as a screening tool for obstructive sleep apnea among different populations: a systematic review and meta-analysis. PLoS ONE. 2015;10(12):e0143697. https://doi.org/10.1371/journal.pone.0143697.
Chiu HY, Chen PY, Chuang LP, Chen NH, Tu YK, Hsieh YJ, Wang YC, Guilleminault C. Diagnostic accuracy of the Berlin questionnaire, STOP-Bang, STOP, and Epworth sleepiness scale in detecting obstructive sleep apnea: a bivariate meta-analysis. Sleep Med Rev. 2017;36:57–70. https://doi.org/10.1016/j.smrv.2016.10.004.
Wang W, Yuan S, Le Grange JM, Zheng H, Yao T, Peng W, Zhang J. Evaluating the performance of five scoring systems for prescreening obstructive sleep apnea–hypopnea syndrome. Sleep Breath. 2020. https://doi.org/10.1007/s11325-020-02227-6.
WHO. Process of translation and adaptation of instruments. World Health Organization. 2007. https://www.who.int/substance_abuse/research_tools/translation/en/. Accessed 7 June 2020.
Gaig C, Iranzo A. Sleep-disordered breathing in neurodegenerative diseases. Curr Neurol Neurosci Rep. 2012;12(2):205–17. https://doi.org/10.1007/s11910-011-0248-1.
Berry RB, Budhiraja R, Gottlieb DJ, Gozal D, Iber C, Kapur VK, Marcus CL, Mehra R, Parthasarathy S, Quan SF, Redline S, Strohl KP, Davidson Ward SL, Tangredi MM. Rules for scoring respiratory events in sleep: update of the 2007 AASM manual for the scoring of sleep and associated events. Deliberations of the sleep apnea definitions task force of the American Academy of Sleep Medicine. J Clin Sleep Med. 2012;8(5):597–619. https://doi.org/10.5664/jcsm.2172.
Fonseca LB, Silveira EA, Lima NM, Rabahi MF. STOP-Bang questionnaire: translation to Portuguese and cross-cultural adaptation for use in Brazil. J Bras Pneumol. 2016;42(4):266–72. https://doi.org/10.1590/s1806-37562015000000243.
BaHammam AS, Al-Aqeel AM, Alhedyani AA, Al-Obaid GI, Al-Owais MM, Olaish AH. The validity and reliability of an arabic version of the STOP-Bang questionnaire for identifying obstructive sleep apnea. Open Respir Med J. 2015;9:22–9. https://doi.org/10.2174/1874306401509010022.
Balsevičius T, Vaitukaitienė G, Šaduikytė B, Miliauskas S, Pribuišienė R. Validating the Lithuanian version of the STOP-BANG questionnaire for diagnosing obstructive sleep apnea. Sleep Breath. 2021. https://doi.org/10.1007/s11325-020-02256-1.
Knol DL, Berger MP. Empirical comparison between factor analysis and multidimensional item response models. Multivar Behav Res. 1991;26(3):457–77. https://doi.org/10.1207/s15327906mbr2603_5.
Lorenzo-Seva U, Ferrando PJ. TETRA-COM: a comprehensive SPSS program for estimating the tetrachoric correlation. Behav Res Methods. 2012;44(4):1191–6. https://doi.org/10.3758/s13428-012-0200-6.
Veasey SC, Rosen IM. Obstructive sleep apnea in adults. N Engl J Med. 2019;380(15):1442–9. https://doi.org/10.1056/NEJMcp1816152.
Chung F, Yegneswaran B, Liao P, Chung SA, Vairavanathan S, Islam S, Khajehdehi A, Shapiro CM. STOP questionnaire: a tool to screen patients for obstructive sleep apnea. Anesthesiology. 2008;108(5):812–21. https://doi.org/10.1097/ALN.0b013e31816d83e4.
Bille J, Bille-Hasselstrom C, Petersen CG. Translation and validation of the STOP-Bang questionnaire for obstructive sleep apnoea into Danish. Dan Med J. 2015;62(12):A5158.
Abdullah B, Idris AI, Mohammad ZW, Mohamad H. Validation of Bahasa Malaysia STOP-BANG questionnaire for identification of obstructive sleep apnea. Sleep Breath. 2018;22(4):1235–9. https://doi.org/10.1007/s11325-018-1663-1.
Duarte RLM, Fonseca LBM, Magalhães-da-Silveira FJ, Silveira EAD, Rabahi MF. Validation of the STOP-Bang questionnaire as a means of screening for obstructive sleep apnea in adults in Brazil. J Bras Pneumol. 2017;43(6):456–63. https://doi.org/10.1590/s1806-37562017000000139.
Randerath W, Bassetti CL, Bonsignore MR, Farre R, Ferini-Strambi L, Grote L, Hedner J, Kohler M, Martinez-Garcia MA, Mihaicuta S, Montserrat J, Pepin JL, Pevernagie D, Pizza F, Polo O, Riha R, Ryan S, Verbraecken J, McNicholas WT. Challenges and perspectives in obstructive sleep apnoea: report by an ad hoc working group of the Sleep Disordered Breathing Group of the European Respiratory Society and the European Sleep Research Society. Eur Respir J. 2018. https://doi.org/10.1183/13993003.02616-2017.
El Shayeb M, Topfer LA, Stafinski T, Pawluk L, Menon D. Diagnostic accuracy of level 3 portable sleep tests versus level 1 polysomnography for sleep-disordered breathing: a systematic review and meta-analysis. CMAJ. 2014;186(1):E25-51. https://doi.org/10.1503/cmaj.130952.
We thank the SBQ Dr. Frances Chung and the University Health Network for allowing us to use the STOP-Bang questionnaire. We also thank the Slovenian Research Agency for their financial support (Grant No. P3-0338).
This study was funded by the Slovenian Research Agency (Grant No.: P3-0338).
Ethics approval and consent to participate
Research was conducted observing the principles of the Declaration of Helsinki and The Oviedo Convention. Consent from the Slovene National Medical Ethics Committee was obtained on Dec 15th 2016 (Ref. No. 651/2016-2).
Consent to participate and publish
All the participants gave their informed written consent to participate in the study and for their data to be used for publication with their identities hidden.
The authors have no conflict of interest to report.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Pangerc, A., Petek Šter, M. & Dolenc Grošelj, L. Translation and validation of the STOP-Bang questionnaire into Slovene. Eur J Med Res 26, 32 (2021). https://doi.org/10.1186/s40001-021-00503-z