Open Science Publication
Mobile applications come along with the advantage of scalability and the possibility of potentially reaching all digitalized parts of society. Their effect on the general public as part of an insurance-covered prevention program has not yet been evaluated.
Users of the App "7Mind" who participated in a mindfulness-based stress management course (ABSM) answered questionnaires about their mental state. This survey-based, pre-post, single-arm design aimed to measure their mindfulness and its effects on the participants. The Freiburg Mindfulness Inventory measured mindfulness.
7117 participants submitted questionnaires. 829 datasets could be analyzed for pre-post differences. The Wilcoxon signed-rank test for paired groups returned significant improvements in Mindfulness, perceived pain, and other variables.
The results promise that meditation apps could play a role in improving mental health across society. However, the drop-out rates demand further investigation of reasons for non-adherence and whether certain social groups benefit less from digital mindfulness interventions than others.
Mindfulness techniques have shown significant effects on healthy individuals and might have positive effects on patients suffering from chronic pain and mental disorders. Mobile applications come along with the advantage of scalability and the potential of reaching all parts of society, also those who cannot attend in-person meditation sessions due to their limitations.
According to a recent metanalysis of mindfulness meditation applications, many papers investigated the influence of App-based MBSR programs on specific groups of people such as students, employees, or cancer patients in an intervention with more than 100 participants. Fewer large studies looked at the general population. This primary prevention measure has exceptional potential compared with programs that only address particular groups for the mental and physical health of the general population. This study investigates whether digital mindfulness-based stress management training offered to the broad society as a health insurance-covered prevention program can affect it positively. Moreover, it is indicated to determine whether all social groups can benefit from such an application to the same degree.
This was a prospective, survey-based, pre/post, single-arm, open-label study. The study aimed to find out whether an App-based intervention can change the Freiburg Mindfulness Inventory (FMI) score significantly within a general population.
No usage data were collected within the smartphone application for this study. A CRM system sent emails with links to the survey platform SurveyMonkey® to all participants after completing the first module (t0) of the ABSM (Mindfulness-based stress management) course. The second questionnaire was exclusively sent out after the final 8th module. This marked t1. The original questionnaires are deposited in the attachment (online only).
A user-created anonymous 4-digit identification code was used to match both datasets. The data was collected by 7Mind® between 13.12.2021 and 26.05.2022. No later responses were considered for this study. Since the course was self-timed, users could start the course independently from the start and end of this data collection.
The course was available to the general public on Android and IOS devices. Inclusion criteria were the download of the App "7Mind" and enrollment for the course. Every user who submitted the questionnaire was included in the study. Therefore, there was no planning concerning the number of participants ahead. There were no exclusion criteria.
All participants submitted the questionnaires voluntarily, agreed to its data procession, and could drop out of the study at any time. The users did not receive compensation for participation. The costs for the course (75€) were - depending on their specific insurance - refunded after completion, or they were covered in advance. This coverage was independent of participation in the surveys. Users who participated in the course also received access to the entire meditation library of the app.
The course encouraged users to deepen their practice with the additional content, but this did not affect the coverage by the insurance.
The course consisted of eight 45-minute-long audio modules. Each included education about stress, mindfulness exercises, and a mindful meditation session. The participants received a handout with a content summary and a quiz about the module after each session via email to continue. Following the guidelines of ZPP (German institution for prevention that certifies insurance covered prevention programs), participants could only absolve one module a week.
The t0 questionnaire assessed 42, and the t1 questionnaire 46 variables. Thirty-six questions were asked at t0 and t1 for pre/post comparison and included the Freiburg Mindfulness Inventory (FMI) and questions recommended by ZPP for evaluating prevention courses. These questions mainly addressed stress and mental well-being. Other questions asked for sociodemographic data and a rating of the course's success. No data about nationality or ethnicity were collected.
The German Freiburg Mindfulness Inventory short form (FMI) was used to measure mindfulness. The original FMI short form is a 14-item assessment that can be used for participants with no experience in mindfulness meditations and was developed by Walach et al. based on the FMI invented by Buchheld et al.. The FMI measures the factors "presence" and "acceptance". Higher scores indicate higher mindfulness.
Following the FMI Rasch analysis results of an item response analysis, item 13 ("I am impatient with fellow human beings.") was left out for improved internal consistency and construct validity. Sauer et al. concluded that the adjusted two-factorial FMI-13 has an acceptable approximation to Rasch requirements. The questionnaire for this study asked about patience with fellow human beings in addition (apart from the FMI score) to cover this subject too.
The CSV file created by SurveyMonkey® was imported into Excel at first to delete personal data such as IP addresses and to blind the variable titles for the analysis. This process was done by the author, who later analyzed the data. The raw data, the codes for unblinding, and the transcript of all R-operations are deposited in the appendix (online only).
The blinded variable ID consisted of a random 4-digit algorithm-generated code and two codes that provided necessary information about the scale level and applicable tests. Participants with only a single time point (only one survey taken or inability to link the forms) were excluded from the confirmatory analysis.
All statistical analyses for individual pre/post comparisons were blinded. The unblinding happened before the FMI score calculation because the calculation required the awareness of what parameters need to be summed. The score was calculated for all participants with matched datasets.
Since the FMI asks questions on a Likert scale, the means of analysis were limited to methods suitable for a discrete, ordinal measurement scale.
All variables with observations at t0 and t1 were compared using the two-sided Wilcoxon signed-rank test for paired groups with continuity correction. The Null-Hypothesis was "There is no significant change between t0 and t1". This provided the confidence intervals and p-values. Alpha was set to 0.05 for all tests using data once. To address the family-wise error, alpha was Bonferroni adjusted for variables involved in multiple tests.
Rstudio Version "2022.02.0+443 "Prairie Trillium" together with the "psych" package and "ggplot2" package were used for the analysis. The analysis code was transcripted in an RMarkdown document and can be seen here (insert a link to the file).
Only users who read the survey invitation email they received could participate in the study. The 4-digit code for identification was based on only three questions, so it was possible that the same code was created more than two times (which led to the inability to match the data of this person in the datasets). Furthermore, it was possible that participants did not provide their code correctly in the second questionnaire at t1, which prevented the matching.
7117 participants submitted the first questionnaire. 2629 sent the second form too. 829 datasets could be matched in RStudio by the user-generated ID.
Participants of the study were predominantly female (76,7%) and highly educated (72,8%), which means they had at least a general qualification for university entrance. The average age was 41 ± 12 years.
82% of the enrollees have not participated in another health prevention course within the last 12 months. The course ratings at t0 indicated a high satisfaction at the end of the program for those who finished it. Bar charts and boxplots of all variables - including pre/post comparisons if applicable - are provided at (add address here).
The gender and educational level-specific non-completion rates were similar apart from participants with a degree lower than secondary school (see table 1). Since people were not asked about gender and education at t1, the only comparison can be between t0 and the merged file (that includes t1 participants combined with the information they gave at t0). It remains unknown how many participants of each gender and educational status submitted the second form but could not be matched.
The primary aim of the study was to find out whether an App-based intervention can change the FMI score significantly within a general population. There was a median shift of the FMI Score from 28 at t0 to 36 at t1. The two-sided Wilcoxon signed-rank test for paired groups returned a significant result that the location shift is not equal to 0 (p < 2.2e-16). The 95 percent confidence interval lies between 7.500078 and 6.500035. The analysis of Cronbach Alpha returned 0.88 at t0 and t1.
The participants rated their state of general health, pain level, and the degree of restriction by pain in daily life significantly lower after the course. All these variables improved significantly. The dataset also includes statements about improved stress management, and other related subjects that cannot be fully addressed here are provided in the appendix.
The primary aim of the study was to find out whether an App-based intervention can change the FMI score significantly within a general population. While the results seem to indicate a clear improvement, they should be interpreted with care due to the limitations of the study design.
All statistical tests turned out to be significant, and all differences pointed in the intended direction of improvement. This confirms that mindfulness-based stress management programs and meditation can also be taught by Apps. The improvements of this prevention course need to be reevaluated after six and twelve months to examine their long-term effects.
Since the course was in German and only German public health insurances cover its costs, it is to assume that most participants lived in Germany, although no data about nationality or ethnicity were collected.
The dominant user group was female and highly educated. This user group might benefit the most while males and less educated social classes might be reached less. This agrees with other findings concerning prevention measures in general and especially with users of meditation apps.
Since a large group of society has never signed up for the course, the gender- and education-specific dropout analysis (which indicates similar non-completion rates for all groups) does not address this issue. The concern appears rather to be how many lower educated people and males sign up than how many complete the program. Nevertheless, it could be that those who have signed up differ from the general public in other variables (such as spiritual interest or similar) that were not measured in this study. Future studies need to investigate the underlying reasons for user group homogeneity and strategies for inclusion of all social groups further.
The study does not provide data that explains the magnitude of the dropout rate. Although the absence of a (voluntarily submitted) t1-questionnaire is not equal to the program's termination, a noticeable non-completion rate is to be expected. The matching process between the datasets of t0 and t1 limits what we can say about those who dropped out. We do not have statistical confidence whether they are significantly different from those that remained. However, at least for women and men and for all educational levels above no degree, the remaining percentages did not vary more than two percent.
Non-adherence is a common limitation of digital mental health interventions. It is reasonable to assume that those who had negative experiences during the program were less likely to finish it. On the other hand, participants' willingness to complete a program might be decreased in the absence of a severe urge to address one's condition. Future studies need to investigate specifically why participants abort courses.
Since Apps have no in-person contact via an instructor, negative experiences and confusion can be less addressed (although users could write emails to the provider in this case). The major advantage of Apps is that they are scalable, easily accessible, cheap for the public health system, and automized. This turns against them in this context.
The pre/post design comes with the virtue that no between-person variability played into the comparison of the groups. However, the absence of a placebo group combined with the open-label design does not control the Hawthorne effect.
There were multiple approaches in mindfulness science to introduce a control group. Cognitive-behavioral therapies, massages, stress management, or stretching exercises were used in different studies to compare placebo probands with participants in meditation intervention groups. Such control measures come along with the limitation that they probably have their own effect on the examined variables. Zeidan et al. have addressed this issue with so-called "Sham Meditations". Sham meditations are based on breathing exercises and the propagation of the belief that participants would be meditating. Participants were not taught how to accept their sensations and thoughts to return to the present moment as in mindfulness exercises. Zeidan et al. have also shown on the fMRI that Sham meditation (along with placebo and book-listening control groups) activates different neural correlates than mindfulness meditation.
Future research of mindfulness prevention courses would ideally have two control groups - one with people who receive instructions similar to the mentioned "Sham Meditation" and one with people who engage in an activity such as audiobook listening. Moreover, there needs to be further research that measures biological markers of stress and its neural correlates instead of only relying on what the participants state.
The observed reduction of perceived pain after the mindfulness program agrees with other findings in that field. There might be neurophysiological explanations for this improvement. These mechanisms seem to be unique, non-opioidergic, and work differently than placebo. May et al. used naloxone as an opioid-antagonist to investigate whether endogenous opioids are responsible for meditation analgesia. The blockage of opioid receptors did even enhance meditations' analgesic effects. The underlying mechanisms for meditations pain-relieving effects are still to be discovered. The fact that participants stated also in this study a reduction of pain underlines the need for further investigation that includes measurements of neurophysiological correlates for pain.
The app intervention has shown a significant effect on the FMI score that pointed in the intended direction. The major caveats are the dropouts of participants in the study and the selective audience that is currently reached by programs comparable to the investigated Mindfulness-based stress management course. Therefore, solutions need to be found to diversify the user group and measures to increase completion rates. Although the limitations of this study and the program need to be considered, application-based interventions appear to be beneficial for their users and need their established place in modern healthcare systems.
R script as R markdown:
Additional values and code explanation:
ABSM course data collection material:
FMI score table:
Please request by emailing [email protected]
Data Availability Statement
This will be filled out after the peer review of the paper.