STUDY QUESTION
What is the inter-/intra-observer agreement and diagnostic accuracy among gynaecological and non-gynaecological ultrasound specialists in the prediction of pouch of Douglas (POD) obliteration (secondary to endometriosis) at offline analysis of two-dimensional videos using the dynamic real-time transvaginal ultrasound (TVS) ‘sliding sign’ technique?
SUMMARY ANSWER
The inter-/intra-observer agreement and diagnostic accuracy for the interpretation of the TVS ‘sliding sign’ in the prediction of POD obliteration was found to be very acceptable, ranging from substantial to almost perfect agreement for the observers who specialized in gynaecological ultrasound.
WHAT IS KNOWN ALREADY
Women with POD obliteration at laparoscopy are at an increased risk of bowel endometriosis; therefore, the pre-operative diagnosis of POD obliteration is important in the surgical planning for these women. Previous studies have used TVS to predict POD obliteration prior to laparoscopy, with a sensitivity of 72–83% and specificity of 97–100%. However, there have not been any reproducibility studies performed to validate the use of TVS in the prediction of POD obliteration pre-operatively.
STUDY DESIGN, SIZE, DURATION
This was a reproducibility study which involved the offline viewing of pre-recorded video sets of 30 women presenting with chronic pelvic pain, in order to determine POD obliteration using the TVS ‘sliding sign’ technique. The videos were selected on real-time representative quality/quantity; they were not obtained from sequential patients. There were a total of six observers, including four gynaecological ultrasound specialists and two fetal medicine specialists. The study was conducted over a period of 1 month (March 2012–April 2012).
PARTICIPANTS/MATERIALS, SETTING, METHODS
The four gynaecological ultrasound observers performed daily gynaecological scanning, while the other two observers were primarily fetal medicine sonologists. Each sonologist viewed the TVS ‘sliding sign’ video in two anatomical locations (retro-cervix and posterior uterine fundus), i.e. 60 videos in total. The POD was deemed not obliterated, if ‘sliding sign’ was positive in both anatomical locations (i.e. anterior rectum/rectosigmoid glided smoothly across the retro-cervix/posterior fundus, respectively). If the ‘sliding sign’ was negative (i.e. anterior rectum/rectosigmoid did not glide smoothly over retro-cervix/posterior fundal region, respectively), the POD was deemed obliterated. Diagnostic accuracy and inter-observer agreement among the six sonologists was evaluated. The same sonologist was also asked to reanalyse the same videos, albeit in a different order, at least 7 days later to assess for intra-observer agreement. A separate analysis of the inter- and intra-observer correlation was also performed to determine the agreement among the four observers who specialized in gynaecological ultrasound. Cohen's coefficient <0 meant that there was poor agreement, 0.01–0.20 slight agreement, 0.21–0.40 fair agreement, 0.41–0.60 moderate agreement, 0.61–0.80 substantial agreement and 0.81–0.99 almost perfect agreement.
MAIN RESULTS AND THE ROLE OF CHANCE
Agreement (Cohen's ) between all six observers for the interpretation of the ‘sliding sign’ for both sets of videos in both regions (retro-cervix and fundus) ranged from 0.354 to 0.927 (fair agreement to almost perfect agreement) compared with 0.630–0.927 (substantial agreement to almost perfect agreement) when only the gynaecological sonologists were included. The overall multiple rater agreement for the interpretation of the ‘sliding sign’ for both video sets and both regions was Fleiss' 0.454 (P-value <0.01) for all six observers and 0.646 (P-value <0.01) for the four gynaecological ultrasound specialists. The multiple rater agreement for all six or all four observers was higher for the retro-cervical region versus the fundal region (Fleiss' 0.542 versus 0.370 and 0.732 versus 0.560, respectively). The intra-observer agreement among the six observers for the interpretation of the ‘sliding sign’ and prediction of POD obliteration ranged from Cohen's 0.60–0.95 and 0.46–1.0 (P-value <0.01), respectively. After excluding the fetal medicine specialists, the intra-observer agreement for the interpretation of the ‘sliding sign’ and the prediction of POD obliteration ranged from Cohen's 0.71–0.95 and 0.67–1.0, respectively, indicating substantial to almost perfect agreement. When comparing the four gynaecological observers for the prediction of POD obliteration using the TVS ‘sliding sign’ (after excluding cases with the POD outcome classified as ‘unsure’ by the observers), the results for accuracy, sensitivity, specificity, positive and negative predictive value were 93.1–100, 92.9–100, 90.9–100, 77.8–100 and 97.7–100%, respectively.
LIMITATIONS, REASONS FOR CAUTION
The ‘gold standard’ for the diagnosis of POD obliteration is laparoscopy; however, laparoscopic data were available only for 24 out of 30 (80%) TVS ‘sliding sign’ cases included in this study. Although this should not affect the inter- and intra-observer agreement findings, the ability to draw conclusions regarding the diagnostic accuracy of the TVS ‘sliding sign’ in the prediction of POD obliteration is somewhat limited. In addition, the diagnostic accuracy findings should be interpreted with the caveat that the cases classified as ‘unsure’ for the prediction of POD obliteration were excluded from the analysis.
WIDER IMPLICATIONS OF THE FINDINGS
We have validated the dynamic real-time TVS ‘sliding sign’ technique for the prediction of POD obliteration, and this simple ultrasound-based test appears to have very acceptable inter-/intra-observer agreement for those who are experienced in gynaecological ultrasound. Given that women with POD obliteration at laparoscopy have an increased risk of bowel endometriosis and requirement for bowel surgery, the TVS ‘sliding sign’ test should be considered in the pre-operative imaging work-up for all women with suspected endometriosis, to allow for appropriate surgical planning. We believe the TVS ‘sliding sign’ technique may be easily learned by sonologists/sonographers who are familiar with performing gynaecological ultrasound, and that further studies are required to confirm the diagnostic accuracy of this new ultrasound technique amongst sonologists/sonographers with various levels of experience.
STUDY FUNDING/COMPETING INTEREST(S)
This study received no specific grant from any funding agency in the public, commercial or not-for-profit sectors and the authors declare no competing interests.
Source:
http://humrep.oxfordjournals.org/cgi/content/short/28/5/1237?rss=1
Recommendation and review posted by G. Smith
