Assessing the Readability and Content-Quality of YouTube Transcripts on Genitourinary Syndrome of Menopause

YouTube has emerged as a popular platform for sharing information, including medical knowledge. Due to its accessibility, user-friendly interface, and extensive reach, this has made it particularly valuable for educational purposes. Approximately 76% of women use social media outlets for obtaining medical and health-related information.¹ However, the abundance of health-related content on YouTube has raised concerns regarding the accuracy and comprehensibility of the available information.² While there have been studies evaluating the quality of YouTube content on urinary tract infections and pelvic organ prolapse, no studies have specifically focused on genitourinary syndrome of menopause (GSM).^2,3

GSM, a relatively new term established in 2014 and previously known as vulvovaginal atrophy, refers to a collection of symptoms that arise from decreased estrogen levels during menopause.⁴ It affects about half of postmenopausal women.⁵ These symptoms encompass various issues related to the genitals and urinary system, such as vaginal dryness, pain during sexual intercourse, urinary incontinence, and recurrent urinary tract infections. Treatment options for GSM involve a broad spectrum, ranging from the use of lubricants and moisturizers to estrogen replacement therapy and the application of newer laser therapies.⁶ Despite the prevalence of GSM, it is often underrecognized and undertreated.⁵

The objective of this study is to assess the readability and content quality of YouTube transcripts on GSM. It may help identify potential information gaps and contribute to the development of accurate and easily understandable resources for women affected by GSM. Ultimately, this can lead to increased conversations with providers on treatment options and greater awareness of this condition.

Methods or Case Description:

In May 2023, “Genitourinary Syndrome of Menopause” was searched on YouTube using an incognito browser. The first 184 videos were identified and filtered based off the following inclusion criteria: Videos < 10 min in length, relevance/ mention of Genitourinary Syndrome of Menopause, and transcript availability in English. After inclusion criteria was applied, 36 videos remained for analysis. The following descriptive data were collected: video length, publication date, and number of views. Video transcripts were downloaded, edited for punctuation and spelling, and then assessed for readability, using an online readability software (https://readabilityformulas.com/free-readability-formula-tests.php), via 3 validated formulas from: Flesch-Kincaid grade level formula, Gunning Fog, and Simple Measure of Gobbledygook (SMOG) readability formula. Assessment of scientific information accuracy and reliability of the videos was rated independently by two reviewers using a modified DISCERN criteria that evaluates the quality of written health information. All statistical analyses were performed using GraphPad Prism 9 Software.

Outcomes:

80% of the videos that fit the inclusion criteria were published within the past 5 years. The median grade level for YouTube transcripts on GSM were 11.10 (Q1=9.2, Q3=13) by Flesch-Kincaid calculations and 10.55 (Q1= 9.05, Q3=12.18) by SMOG calculations indicating a requirement of a 10^th grade reading level or higher. The percentage of Flesch-Kincaid and SMOG scores that were at or below the 8^th grade reading level were 22% and 16% respectively. Gunning Fog output revealed that the median transcript readability was close to “difficult” 14.35 (Q1=12.33, Q3=16.58).

When assessing for reliability, nearly all the YouTube videos had aims that were clear and achieved. 86.11% of the videos featured discussion by medical professionals/ GSM researcher. Nearly all the YouTube videos on GSM presented information that is based on science and intended to educate using valid evidence and concepts. Over 90% of the videos did not present misinformation. Further, over 90% of the videos utilized credible sources with personal or medical expertise and knowledge and came from reliable sources of information. 56.94% of the videos provided partial to no additional sources of information for patients to refer to. Similarly, 38.89% of the videos failed to or partially mentioned areas of uncertainty. An increase in video length was positively correlated with the mention of areas of uncertainty (R² = 0.3424, p = 0.0002). The average median video duration for videos ranked 5 for mention of areas of uncertainty was 7.53 minutes.

Conclusion:

The present study suggests that a majority of the current YouTube videos on GSM are scientifically accurate, reliable, and provide minimal misinformation on GSM published on YouTube. This can be attributed a large number of the videos involving discussion by medical professionals. Further, due to the specialized nature of the term "GSM", individuals without medical expertise may have limited familiarity with it, thus resulting in a scarcity of videos produced by non-medical individuals on this subject.

The lack of additional sources of information in a significant portion of the videos and the limited mention of areas of uncertainty highlight areas for improvement in terms of depth and transparency. While a longer video duration increases time for discussion of these two areas, a video should be less than 10 minutes, as user engagement declines after 10 minutes.⁷ From our results, a video less than 6 minutes may not be sufficient for discussing uncertainty of GSM and may render the video unreliable. Our findings also revealed that none of the YouTube transcripts achieved a “readable” score as determined by the Gunning Fog Formula. Additionally, the readability level of the transcripts generally required a higher reading proficiency. A large portion of the videos did meet the recommended 6^th grade reading level for patient material, as recommended by the NIH.⁸ This emphasizes the need for enhanced development of materials to effectively communicate with patients with low health literacy. Future evaluation of information on GSM via other social media platforms such as TikTok and Reddit is also warranted.