Evaluating the quality, safety, and functionality of commonly used smartphone apps for bipolar disorder mood and sleep self-management
International Journal of Bipolar Disorders volume 10, Article number: 10 (2022)
Individuals with bipolar disorder (BD) are increasingly turning to smartphone applications (apps) for health information and self-management support. While reviews have raised concerns regarding the effectiveness and safety of publicly available apps for BD, apps surveyed may not reflect what individuals with BD are using. The present study had two aims: first, to characterize the use of health apps to support mood and sleep amongst people with BD, and second, to evaluate the quality, safety and functionality of the most commonly used self-management apps.
Findings illustrate the potential of smartphone apps to increase the reach of psychosocial interventions amongst people with BD. Apps were largely created by commercial developers and designed for the general population, highlighting a gap in the development and dissemination of evidence-informed apps for BD. There may be risks in using generic health apps for BD self-management; clinicians should enquire about patients’ app use to foster conversations about their particular benefits and limitations.
Bipolar disorder (BD) is a chronic, severe mental disorder characterized by the experience of episodes of clinically depressed or elevated mood states. The lifetime prevalence of bipolar spectrum disorders is estimated to be 2.4% globally (Merikangas et al. 2011), and the condition is ranked as the 5th leading cause of disease burden among mental disorders by the World Health Organization (Ferrari et al. 2016). Self-management is recognized as central to living well with BD by both treatment guidelines (Kendall et al. 2016; Malhi et al. 2020; Yatham et al. 2018) and people living with the disorder (Michalak et al. 2016; Murray et al. 2011). Particular emphasis is placed on detecting and responding to changes in two symptom domains, mood and sleep, as there is evidence that psychosocial therapies which incorporate these strategies can reduce symptoms and prevent relapse/recurrence (Miklowitz and Scott 2009). However, distance, cost, stigma, and trust in the healthcare system present barriers to accessing clinician support for self-management practices (Leitan et al. 2015).
Digital health interventions, particularly smartphone applications (apps; otherwise known as mHealth), may improve access to evidence-based psychosocial strategies for BD. There are a number of promising developments in mHealth research for BD (Torous and Brady 2020), such as employing smartphone passive data-sensing capabilities (Faurholt-Jepsen et al. 2013; Matthews et al. 2016; Beiwinkel et al. 2016), facilitating clinician review of self-monitoring data (Faurholt-Jepsen et al. 2019), and delivering psychoeducation programs (Hidalgo-Mazzei et al. 2016; Depp et al. 2015). Despite such remarkable progress in leveraging new technologies to better monitor BD symptoms and deliver interventions, evidence-supported apps are slow to be made publicly accessible due to research-to-practice translational lags and complex regulatory systems (Mathews et al. 2019). Further, research grants rarely cover the costs required for ongoing customer/technical support, and as a consequence many academic mHealth projects do not reach commercial marketplaces (Jake-Schoffman et al. 2017). In contrast to the slow pace of development, evaluation and dissemination in academic contexts, an increasingly large number of apps claiming to support mental health needs are available for download via commercial marketplaces (Torous et al. 2018). Given that internet-connected device ownership and interest in smartphone-based interventions is high amongst people living with BD (Hidalgo-Mazzei et al. 2019), it is likely that consumers with BD are already self-selecting from available mental health apps. Accordingly, there is widespread concern about the quality, safety and efficacy of apps used by consumers (Torous et al. 2018; Larsen et al. 2019; Wisniewski et al. 2019), as individuals may be downloading and using mental health apps that lack evidence bases and without consulting their healthcare provider.
Although such reviews have elucidated the type and quality of publicly available mental health apps (including those marketed for BD), it has been noted that marketplace searches alone may not reflect the apps used in real-world contexts (Rubanovich et al. 2017). Despite the proliferation of health apps, the majority of downloads are accounted for by a small number of apps (Aitken and Lyle 2015), and download metrics may not reflect longer-term uptake. Search results are personalized based on user characteristics (Grundy et al. 2016), and popularity metrics may be biased by developers paying for downloads/positive reviews (BinDhim et al. 2015). Indeed, it has been noted that the most downloaded mental health apps are not necessarily the ones with the most active users (Carlo et al. 2020).
Conclusions drawn on the basis of marketplace searches may therefore not accurately represent real-world use. For example, while a recent review of apps for depression or anxiety reported that evidence-supported treatment components for these conditions were poorly represented (Wasil et al. 2019), when the appraisal was weighted by the number of active users, the reach of specific treatment elements changed. Although mindfulness exercises were present in only 37% of reviewed apps, this technique was encountered by the largest proportion (96%) of monthly active users. While this analysis better depicts naturalistic usage, it has limitations in representing apps used by those with mental health needs. First, user counts included downloads by healthy adults. Second, apps were identified by searching for “depression” and “anxiety”; yet people with mental health needs have reported using wellbeing apps designed for the general population, not necessarily those targeting their diagnosed conditions (Rubanovich et al. 2017; Beard et al. 2019). To accurately appraise the quality of apps adopted by people with BD in real-world contexts, we must directly ask the individuals in question to describe their mHealth use (Rubanovich et al. 2017).
The present survey aimed to identify apps used by people with BD to self-manage two core symptom domains (mood and sleep), and to evaluate their quality and features according to a standardized framework (Lagan et al. 2020b).
An overarching, international, cross-sectional online survey investigated the use of smartphone apps to support mental health and quality of life, attitudes towards app features, privacy concerns, and digital health literacy amongst people living with BD. The survey was conducted according to a community-based participatory research framework: the Collaborative RESearch Team to study psychosocial issues in Bipolar Disorder (CREST.BD) research network community advisory group (comprised primarily of people living with BD) was consulted regularly regarding survey content and recruitment strategies. This community-based participatory research process is described in full elsewhere (Morton et al. 2021a). Prior reports have summarized predictors of digital health literacy (Morton et al. 2022) and endorsement of various app features/privacy concerns in this sample (Morton et al. 2021a). Here, to characterize use of self-management apps specifically, we detail responses to survey items related to apps used in self-management of mood and sleep.
Questionnaires were administered via Qualtrics. Data collection occurred between February and July 2020. Ethics approval was granted by the University of British Columbia Behavioral Research Ethics Board. All participants received written information about the study and indicated their consent before proceeding. Participants who exited the survey prior to completion were treated as having withdrawn consent, and their data were not analyzed. Data in the study were treated confidentially and survey responses stored on a secure server in Canada.
Participants and recruitment
Participant recruitment occurred via promotion on CREST.BD social media pages, paid advertisements on Facebook, Instagram, and Twitter, emails to the CREST.BD mailing list and healthcare providers or organizations associated with the CREST.BD network, and featured at online (webinar) and in-person educational events (held in Vancouver, Canada) for individuals with BD. To address initial overrepresentation of white North American participants, social media advertisements specifically targeted users in Asia, Africa, and South America with interests related to BD (i.e., liking social media pages for the International Bipolar Foundation, BP Magazine for Bipolar, and the Depression and Bipolar Support Alliance). Participants were entered into a prize draw for one of two $50 (CAD) Visa gift cards. Inclusion criteria were: (1) age 19 or older, and (2) a self-reported diagnosis of BD.
An online survey was developed on the basis of previous literature and stakeholder input (Additional file 1). Individuals were asked to describe their use of apps to track or support their symptoms and quality of life across a range of domains identified as important to people with BD (Michalak and Murray 2010).
Given concern about the type and quality of apps used to support symptom management in BD (Nicholas et al. 2015; Lagan et al. 2020a), the present analysis focused on items concerning the use of apps related to the core foci of BD self-management (i.e., mood and sleep). Individuals were asked if they used apps to a) track or manage their mood or b) support their sleep/wake routine or sleep quality. If a respondent indicated yes, they were asked to name the apps used and describe their frequency of use. Evaluation of apps used to support other life domains was beyond scope of this analysis.
Use of self-management apps to support mood or sleep
Descriptive statistics were used to characterize the sample and summarize the use of smartphones and self-management apps. One author (LL) categorized free-text responses indicating names of apps used; internet searches were used to confirm the spelling and existence of apps referenced.
Evaluation of commonly used self-management apps
The five most commonly nominated mood and sleep self-management apps were subject to further analysis. This cutoff was selected to ensure the evaluation process was feasible and swift, given concerns regarding the currency of app evaluation papers (Larsen et al. 2020), and was chosen for its similarity to the number of apps reported by prior studies of the most frequently named apps used by people with depression/anxiety (Rubanovich et al. 2017), and a survey of health apps used by the general population (Krebs and Duncan 2015).
The MIND framework
To support comparability with a recent review of apps identified by marketplace searches of the term “bipolar” (Lagan et al. 2020a), we used the Mhealth Index and Navigation Database (MIND) framework (Lagan et al. 2020b) based on the American Psychiatric Association’s (APA) app evaluation model. The initial APA framework was developed through consultation with service users, clinicians, and informaticists (American Psychiatric Association 2017), and amalgamated questions from 45 existing evaluation frameworks (Henson et al. 2019). The MIND framework consists of 105 objective questions that map onto the APA framework and principles of medical ethics, and is divided into six sections: app origin and functionality (background information and accessibility), inputs and outputs (what types of data are collected and generated), privacy and security (data use and sharing, patient safety), clinical foundation (the existence of peer reviewed literature describing the feasibility and efficacy of reviewed apps), features and engagement style (app interface and overall functionality), interoperability and data sharing (ability to export/share data for personal or clinical use). The MIND framework has been used to describe the quality and safety of mental health apps for schizophrenia (Lagan et al. 2021a), bipolar disorder (Lagan et al. 2020a), Spanish language speakers (Muñoz et al. 2021), peripartum affective disorders (Feldman et al. 2021), and top-returned apps for mental health related searches (Lagan et al. 2021b).
It should be noted that as the MIND framework is intended for use by non-specialists (including people with lived experience, healthcare administrators, and policymakers) to provide a general assessment of the quality of a given app, a formal systematic review is not required to answer items in the clinical foundation section, nor are raters instructed in evaluating the quality of scientific literature (Lagan et al. 2021b). In a similar vein, the MIND framework generates a descriptive summary of app characteristics; no total score or quantitative summary is provided. In practice, clinicians are encouraged to conduct a personalized assessment of the fit between a reviewed app and an individual’s needs/priorities (Torous et al. 2018).
All raters (LL, MC, RK, LY) underwent publicly available training in the framework questions (apps.digitalpsych.org). Following training, raters applied the framework questions to two apps that were not the subject of the present investigation (7Cups and ACT iCoach) to ensure reliability with expert-generated criterion ratings. Discrepancies were addressed via discussion with the framework developers. All raters met the minimum threshold for very good interrater reliability, defined as Cohen’s kappa statistic > 0.75 (McHugh 2012), and percentage agreement with criterion ratings was high (> 80%).
Evaluation procedures in the present study
Each app was assessed by at least two raters. Raters used their personal devices to download and test apps. All apps were evaluated between January and April, 2021.
Raters identified peer-reviewed studies by searching Google Scholar and Medline using the app name and “feasibility”, “efficacy” or “effectiveness” as keywords. The reference lists of identified studies were also reviewed for additional relevant literature. To support the feasibility and relevance of the literature review, efficacy studies describing unrelated physical health outcomes (e.g., weight, caloric intake, physical activity) were excluded. This decision was made following a preliminary search for literature related to the Fitbit app, in which numerous studies related to other functions of this app and associated wearable device were identified.
Discrepancies between raters were resolved via consensus discussion. Descriptive statistics were used to summarize key findings from the resulting data.
A total of 919 people with BD responded to the online survey (see Table 1 for demographic details). 81.3% of participants completed the survey between June 21 and July 20. Participants were primarily female (77.9%, n = 716), white (61%, n = 560), residing in North America (43.2%, n = 397), with a mean age of 36.9 years (SD = 12). Approximately half the sample self-reported a BD-II diagnosis (51.9%, n = 477), and most participants were receiving psychiatric treatment, including medication (81.1%, n = 745) and counselling (47.7%, n = 438). The sample was largely employed (55.3%, n = 508), married or in a committed relationship (49.5%, n = 455), and had completed some form of post-secondary education (77.4%, n = 711). Almost all participants reported using apps on a smartphone (97.5%, n = 896).
Use of self-management apps to support mood or sleep
In total, 41.6% (n = 382) of the sample reported using a self-management app related to mood and/or sleep. Specifically, 24.8% (n = 228) of respondents reported use of an app to monitor or improve mood, and 26.3% (n = 242) reported use of an app to monitor or improve sleep (see Table 2). Of individuals using self-management apps, 23% (n = 88) reported using apps related to both mood and sleep. Daily use of self-management apps was common.
A broad spectrum of apps were nominated in relation to each self-management domain. A total of 110 unique apps were described as being used to monitor or improve mood, however the majority were nominated only once (68.2%, n = 75). Similarly, while 104 apps were named in relation to monitoring or promoting optimal sleep, most sleep apps were nominated only once (73.1%; n = 76).
Evaluation of commonly used self-management apps
The top five apps related to mood were nominated a total of 130 times, with the most popular mood app (Daylio) nominated by 59 respondents. Other commonly nominated mood apps included Bipolar eMoods Tracker (n = 45), iMood Journal (n = 9), Moodpath (n = 9), and Calm (n = 8). The top five sleep-related apps were nominated a total of 112 times, with the most popular app (Fitbit) mentioned 50 times. Other commonly nominated sleep apps included Calm (n = 24), Sleep Cycle (n = 15), Samsung Health (n = 14), and Headspace (n = 9). 25 people reported using the alarm function of the pre-installed clock app to support their sleep—as this a function of the smartphone itself, we chose to exclude it from further analysis.
As one app (Calm) appeared in the top five most commonly nominated apps in relation to both mood and sleep self-management, a final set of nine apps was evaluated using the MIND framework. The summary below highlights trends across the reviewed apps. For a full description of reviewed apps according to the 105 framework questions, see Additional file 2. To provide readers with an accessible, brief overview of findings, key characteristics of the nine reviewed apps are also presented in Table 3.
Origin and accessibility
At the time of this analysis, all reviewed mood and sleep apps were available on both the Apple iOS and Google Play stores, and had been updated in the last 180 days—a metric associated with user satisfaction and safety (Wisniewski et al. 2019). Reviewed apps appeared to be popular, with an average rating of 4.6/5 on the iOS store. All reviewed apps were commercial; no developers were affiliated with university, healthcare, or government organizations. All but one app was free to download, although the majority (n = 8) required in-app purchases to access all content. Only a small proportion of apps (n = 2) required a continuous internet connection. All apps were available in a language other than English, and the majority (n = 6) had one or more accessibility features (e.g., adjustable text size or text-to-voice).
Inputs and outputs
Reviewed mood and sleep self-management apps most commonly collected user-input data via surveys (n = 7) and diaries (n = 7). Few (n = 3) received data from external devices (e.g., wearables). App outputs typically included notifications (n = 9) or graphs of data (n = 8); few had the option to share information to the user’s social network (n = 4) or contained a link to formal care or coaching (n = 1).
Privacy and confidentiality
Raters did not identify any potentially harmful content in the reviewed apps. The majority of mood and sleep self-management apps (n = 6) had published evidence regarding their feasibility/usability, largely in healthy adults (summarized in Additional file 3). Few apps nominated in relation to mood (n = 2) had evidence to support their efficacy, compared to apps nominated in relation to sleep (n = 4). Existing efficacy studies provided some support for developers’ claims that app use could result in improved mindfulness, stress management, sleep quality, and mood; the design, sample, and findings of these studies are summarized in Additional file 4. However, only one efficacy study included a BD sample.
Features, engagement style, and data sharing
Apps nominated in relation to mood typically included the following features: mood tracking (n = 5), sleep tracking (n = 3), and journaling (n = 5). Apps nominated in relation to sleep typically included the following features: sleep tracking (n = 3), journaling (n = 3), mindfulness (n = 4) and deep breathing (n = 3) exercises, and physical health exercises (n = 3). Reviewed self-management apps were typically designed for self-monitoring rather than the delivery of psychosocial interventions: no mood app was observed to offer elements of cognitive behavior therapy, acceptance and commitment therapy, or dialectical behavior therapy. Two sleep apps were observed to offer elements of cognitive behavior therapy for insomnia.
To support engagement, the majority of both types of self-management apps incorporated audiovisual elements (n = 6), and all incorporated user-generated data (n = 9). However, peer support (n = 1) and gamification (n = 4) were rarely offered. Most apps (n = 8) allowed users to email or export their data, although only three were designed for use with a clinician in conjunction with a treatment plan.
The present study used a large-scale, international web-based survey to identify and evaluate apps used to support mood and sleep, two key foci of self-management in BD. Use of health and wellbeing apps was moderately prevalent: 42% of the sample reported using an app to support self-management of either mood or sleep. These individuals reported a high frequency of mHealth use, suggesting apps were perceived as helpful and engaging. Interestingly, although a number of apps offered both mood and sleep tracking, the reviewed apps were generally nominated in relation to one purpose only. Only Calm and Headspace were nominated in relation to both sleep and mood. The analysis of the quality, safety, and efficacy of the top five sleep and mood self-management apps nominated by people with BD (for a total of nine reviewed apps) offers some reassuring findings, as well as some cautions.
A body of evidence supports the feasibility and acceptability of included apps (Additional file 3), largely in healthy adults. However, extrapolation of this data is limited by the short-term nature of these studies, with the longest follow-up period being eight weeks. Additionally, the presence of reminders from the research team could encourage app uptake beyond what might be expected in a naturalistic setting (Baumel et al. 2019). Individuals with BD may engage, disengage, and re-engage with apps according to the fluctuating course of illness (O'Brien et al. 2020). Although one BD-specific app (eMoods Bipolar Mood Tracker) was evaluated, this was conducted with a sample of healthy volunteers (Tena-Cucala and Cobo 2019). Only one qualitative study of the Headspace app included people with a BD diagnosis (Mistler et al. 2017). Thus, these studies may not be an accurate reflection of the likelihood of engagement in a non-study setting for those living with mental illness. While participants in the present study were asked about their frequency of app use, they were not asked about the length of time since app download. Research on mHealth apps more broadly suggests that sustained engagement in real-world contexts is rare (Fleming et al. 2018). Naturalistic, long-term research in this population is therefore needed to better determine app usage patterns, as available research regarding adherence to apps developed for BD suggests highly heterogenous rates of use (Patoz et al. 2021). Feasibility research is also necessary to identify and address accessibility barriers unique to people with BD, who may experience cognitive difficulties (Bonnín et al. 2021), hand tremors (Gitlin 2016), and fluctuations in attention and motivation in accordance with mood state (Urošević et al. 2008), that together influence app usage.
Evidence for the efficacy of apps is varied (Additional file 4). While the impacts of some apps (i.e., Headspace and Fitbit) have been extensively researched, others had no or little peer reviewed evidence (i.e., Daylio Journal, eMoods Bipolar Mood Tracker, iMood Journal, MoodPath and Sleep Cycle). Although a formal systematic review of the literature and evaluation of study quality was outside scope of this analysis (nor is it required as part of the MIND framework app evaluation), it is worth noting broad trends in the identified research: while some randomized control trials had reasonably large samples, many others had an inactive control, used a single-arm design, and had small sample sizes. Further, the use of healthy volunteers may introduce a sampling bias, as those who have an interest in mHealth may be more likely to engage in these studies. Finally, while some of the outcomes assessed are directly relevant to BD (e.g., mood and sleep quality), many of the studies measured variables related to general wellbeing, such as mindfulness, stress, and attention, and only one qualitative study included people with BD (Mistler et al. 2017). Further research is required to more confidently state the potential efficacy of such app-based interventions in BD populations. In addition, the possibility of harm resulting from use of these apps in people with BD, such as the potential of symptom monitoring exacerbating negative affect, is also not known (Nicholas et al. 2019; Faurholt-Jepsen et al. 2015).
Notably, the most popular mood and sleep apps used by people with BD are not condition-specific, rather, they are ranked among the most prominent health and wellbeing apps for the general population. For example, Fitbit is one of the most commonly installed health and fitness apps amongst US mobile owners (Krebs and Duncan 2015); similarly, Calm, Headspace, and Daylio rank among the most commonly downloaded mental health apps (Carlo et al. 2019; Woodward et al. 2020). While the apps most frequently nominated in this survey were found to largely meet minimum standards for safety and privacy, previous reviews have noted limitations regarding adherence to clinical guidelines for both mood and sleep apps (Nicholas et al. 2015; Lagan et al. 2020a; Grigsby-Toussaint et al. 2017; Qu et al. 2020), and their efficacy for improving outcomes in a BD population is unknown. Importantly, generic health apps may recommend strategies with the potential for harm in a BD population, or fail to detect early warning signs unique to this group. For example, sleep restriction is a technique that can be used to address insomnia in the general population, however this should be used with caution in BD due to the potential for triggering mania (Morton and Murray 2020). Similarly, unidimensional self-report mood scales (i.e., sad to happy) may inappropriately reinforce problematic mood elevation in BD. Apps designed for the general public or unipolar depression may offer suggestions to increase activity in response to low mood; people with BD should be warned that such strategies, while helpful, can trigger or exacerbate manic symptoms. In the absence of widely adopted, evidence-supported, condition-specific apps for BD, clinicians play an important role in supporting patients to recognize and work around the limitations of generic health and wellbeing apps.
Although use of specific apps by people with BD has not previously been investigated, a survey of physical and mental health apps used by people with depression and anxiety found similar trends (Rubanovich et al. 2017). This sample (n = 176) was specifically recruited to test the IntelliCare suite of mental health apps, and as such, findings may not have been reflective of the population of mental health app users more generally. Even so, some similarities with the present analysis were observed: the most commonly used health apps included ones designed for the general population, such as Fitbit, Samsung Health, and Headspace. Of note, of the 139 unique health apps mentioned, 92.8% had less than 5 mentions, and 69.8% had only one mention. This accords with the present study, where 68.2% of named mood apps and 73.1% of named sleep apps were nominated by only one person. Analysis of the rate of download of health apps more broadly suggests that the majority of apps are downloaded less than 5000 times (Aitken and Lyle 2015). Together, these findings highlight a large degree of interindividual variability in the precise apps used to manage health concerns. As such, while the present findings offer some confidence in regards to the quality and safety of the most commonly used apps, clinicians should still be alert to (and enquiring of) the possibility that patients are using poorer quality apps (Aref-Adib et al. 2020).
The preponderance of generic health and wellbeing apps used by respondents in the present study (and the large number of people reporting daily use of such health apps) raises a number of considerations for the development and dissemination of mHealth interventions for BD. Firstly, it is necessary to weigh whether user priorities for aesthetics and functionality are being sufficiently addressed in BD-specific apps: indeed, a qualitative analysis of public reviews of apps developed for BD highlighted a number of unmet needs and complaints regarding functionality (Nicholas et al. 2017a), suggesting that consumer preferences are currently not adequately addressed. This emphasizes the need for research-led app development projects to solicit potential user perspectives during the design phase, as clinicians/researchers and the public may not share priorities for content, features, or aesthetics (Carlo et al. 2019). Secondly, this finding highlights the need for researchers to consider scale-up and commercialization as part of end-of-grant planning (Hidalgo-Mazzei et al. 2020). More research must be conducted into effective dissemination strategies for evidence-informed mHealth interventions. People with mental health conditions (including those in the present study) primarily rely on word of mouth or app store ratings/reviews to identify apps (Rubanovich et al. 2017; Morton et al. 2022; Schueller et al. 2018), which may have contributed to the popularity of generic health and wellbeing apps seen here. Training initiatives may be required to support people with BD in identifying and selecting the most appropriate apps for their specific health needs (Hoffman et al. 2020). Such efforts should be complementary to training and education for healthcare providers; although they may be a valuable source of information on apps, a survey found that only 50% of clinicians have discussed or suggested apps to patients with BD (Morton et al. 2021b). Clinicians who had not discussed apps commonly cited lack of knowledge regarding safe and appropriate apps for this population as a barrier. Finally, the potential negatives of condition-specific apps also require further consideration; there is evidence that symptom monitoring in BD can be experienced as an unpleasant reminder of illness (Bendegem et al. 2014), and potential app users may also be concerned about unintended disclosure through titles and icons that contain references to the diagnosis (Nicholas et al. 2017b). Ideally, awareness raising could encourage developers of popular health apps to include content tailored for this mood disorder, much in the way that generic stress management apps often contain some psychoeducation about clinical depression or anxiety. Indeed, new initiatives targeting transdiagnostic symptoms in psychiatric disorders, in which intervention content is tailored to daily assessments, have shown positive results (Ben-Zeev et al. 2018).
Strengths and limitations
A number of limitations to the present study should be noted. Firstly, our sample was predominantly female, white, employed, and had completed some form of post-secondary education. As such, findings may not generalize to individuals with BD more broadly. Further, estimates of app use may be higher amongst respondents to a web-based survey relative to community samples, although rates of smartphone ownership are similar to other large-scale BD surveys (Hidalgo-Mazzei et al. 2019). Our survey findings may also be reflective of increased uptake of digital health tools associated with the COVID-19 pandemic (Sorkin et al. 2021). Participants were informed that a BD diagnosis was required to be eligible for the survey, and were given the option to self-report a BD subtype. As diagnosis was not confirmed with a clinical interview, it is possible that individuals who did not meet diagnostic criteria for BD may have completed the survey, and findings should be interpreted with requisite caution. Although there is limited research on the characteristics of individuals who self-identify as having BD, one study reassuringly supports that people who self-identify as having BD typically meet diagnostic criteria (Kupfer et al. 2002): a random sample of people who registered to join a BD care registry (n = 100) found that 93% met DSM-IV criteria for lifetime BD following a face-to-face structured clinical interview. In addition, rates of survey completion were likely enhanced by not including a requirement to complete a clinical interview, which would have added significantly to response burden, and detracted from study feasibility (in terms of staff, budget, and time required). On balance, a strength of our study is the large sample size and international representation, given that app use may vary by region (Woodward et al. 2020), with 56.5% of the sample residing outside of North America.
Secondly, there is no consensus approach to evaluating mHeath apps (Torous et al. 2018). A recent scoping review found the MIND categories and questions largely overlap with content encompassed in other existing app evaluation frameworks, suggesting good coherence (Lagan et al. 2021); however, some aspects of user experience (e.g., usability and aesthetic appeal) are not addressed. It has been suggested that the subjectivity inherent in rating such aspects is a likely source of inconsistencies in app evaluations (Carlo et al. 2019). Given this, a strength of using the MIND framework is that it focuses on objective criteria; use of this framework also permits comparison between our analysis of user- nominated apps and earlier work on marketplace offerings for BD (Lagan et al. 2020a).
Our evaluation of apps identified in the survey was not intended as an endorsement of any specific apps for clinical purposes; rather we intended to provide a snapshot of the quality, safety, and efficacy of apps commonly used by people with BD. As such, we chose to focus on apps used to support two core foci of self-management in BD (mood and sleep). Self-management in BD encompasses a broad range of strategies (Murray et al. 2011), and the present analysis may not address all potential targets of apps in BD, such as crisis management/safety planning, medication reminders, fitness and nutrition, amongst others (Murnane et al. 2016). Although we asked participants to describe their use of apps related to other QoL domains, these were not subject to further formal analysis for reasons of feasibility. Additionally, the MIND evaluation framework was developed to describe mental health apps – there may be specific risks, privacy standards, or features which should be described to comprehensively evaluate apps designed for other purposes (e.g., social networking, financial management). However, a strength of this focused approach means our review was less likely to be impacted by the high turnover of marketplace offerings identified as problematic in previous publications (Larsen et al. 2020); indeed, we note that at the time of submission, all reviewed apps remained available in the Google Play and Apple app marketplaces. Finally, although we assessed privacy policies of the included apps, data protections and use as written does not necessarily reflect true practices (Huckvale et al. 2019).
Taken together, these findings reinforce the importance of contextualizing app evaluations according to real-world usage rates, through demonstrating that the earlier reviews of the BD app landscape are not necessarily reflective of apps most commonly used in real-world contexts. Rather, people with BD tended to use apps designed for the general population to support and monitor their mood and sleep. The presence of some privacy protections and evidence regarding app efficacy in healthy adults offers some reassuring findings in regards to quality of apps used by people with BD. However, there is a paucity of evidence exploring benefits and harms of use of these apps specific to BD which requires further consideration. Research-led mHealth development projects should consider how user-centered design and effective dissemination plans may increase the reach and uptake of apps.
Availability of data and materials
Data are not publicly available in accordance with ethics approval given by the ethics board from the participating university. Interested investigators may submit inquiries to the corresponding author.
Collaborative RESearch Team to study psychosocial issues in Bipolar disorder
Aitken M, Lyle J. Patient adoption of mHealth: Use, evidence and remaining barriers to mainstream acceptance. 2015. IMS Institute for Healthcare Informatics.
American Psychiatric Association. App Evaluation Model. 2017. Available from: https://www.psychiatry.org/psychiatrists/practice/mental-health-apps/app-evaluation-model.
Aref-Adib G, Landy G, Eskinazi M, Sommerlad A, Morant N, Johnson S, et al. Assessing digital risk in psychiatric patients: mixed methods study of psychiatry trainees’ experiences, views, and understanding. JMIR Ment Health. 2020;7(7):e19008.
Bauer M, Glenn T, Monteith S, Bauer R, Whybrow PC, Geddes J. Ethical perspectives on recommending digital technology for patients with mental illness. Int J Bipolar Disord. 2017;5(1):6.
Baumel A, Edan S, Kane JM. Is there a trial bias impacting user engagement with unguided e-mental health interventions? A systematic comparison of published reports and real-world usage of the same programs. Transl Behav Med. 2019;9(6):1020–33.
Beard C, Silverman AL, Forgeard M, Wilmer MT, Torous J, Bjorgvinsson T. Smartphone, social media, and mental health app use in an acute transdiagnostic psychiatric sample. JMIR Mhealth Uhealth. 2019;7(6):e13364.
Beiwinkel T, Kindermann S, Maier A, Kerl C, Moock J, Barbian G, et al. Using smartphones to monitor bipolar disorder symptoms: a pilot study. JMIR Ment Health. 2016;3(1):e2.
Ben-Zeev D, Brian RM, Jonathan G, Razzano L, Pashka N, Carpenter-Song E, et al. Mobile health (mHealth) versus clinic-based group intervention for people with serious mental illness: a randomized controlled trial. Psychiatr Serv. 2018;69(9):978–85.
BinDhim NF, Hawkey A, Trevena L. A systematic review of quality assessment methods for smartphone health apps. Telemed J E Health. 2015;21(2):97–104.
Bonnín CdM, Solé B, Reinares M, García-Estela A, Samalin L, Martínez-Arán A, et al. Does cognitive impairment in bipolar disorder impact on a SIMPLe app use? J Affect Disord. 2021;282:488–94.
Carlo AD, Hosseini Ghomi R, Renn BN, Areán PA. By the numbers: ratings and utilization of behavioral health mobile applications. NPJ Digit Med. 2019;2(1):54.
Carlo AD, Hosseini Ghomi R, Renn BN, Strong MA, Areán PA. Assessment of real-world use of behavioral health mobile applications by a novel stickiness metric. JAMA Netw Open. 2020;3(8):e2011978.
Depp CA, Ceglowski J, Wang VC, Yaghouti F, Mausbach BT, Thompson WK, et al. Augmenting psychoeducation with a mobile intervention for bipolar disorder: a randomized controlled trial. J Affect Disord. 2015;174:23–30.
Faurholt-Jepsen M, Vinberg M, Christensen EM, Frost M, Bardram J, Kessing LV. Daily electronic self-monitoring of subjective and objective symptoms in bipolar disorder: The MONARCA trial protocol (MONitoring, treAtment and pRediCtion of bipolAr disorder episodes): a randomised controlled single-blind trial. BMJ Open. 2013;3(7):e003353.
Faurholt-Jepsen M, Frost M, Ritz C, Christensen EM, Jacoby AS, Mikkelsen RL, et al. Daily electronic self-monitoring in bipolar disorder using smartphones—the MONARCA I trial: a randomized, placebo-controlled, single-blind, parallel group trial. Psychol Med. 2015;45(13):2691–704.
Faurholt-Jepsen M, Torri E, Cobo J, Yazdanyar D, Palao D, Cardoner N, et al. Smartphone-based self-monitoring in bipolar disorder: Evaluation of usability and feasibility of two systems. Int J Bipolar Disord. 2019;7(1).
Feldman N, Back D, Boland R, Torous J. A systematic review of mHealth application interventions for peripartum mood disorders: trends and evidence in academia and industry. Arch Womens Ment Health. 2021;2021:1–12.
Ferrari AJ, Stockings E, Khoo JP, Erskine HE, Degenhardt L, Vos T, et al. The prevalence and burden of bipolar disorder: findings from the Global Burden of Disease Study 2013. Bipolar Disord. 2016;18(5):440–50.
Fleming T, Bavin L, Lucassen M, Stasiak K, Hopkins S, Merry S. Beyond the trial: Systematic review of real-world uptake and engagement with digital self-help interventions for depression, low mood, or anxiety. J Med Internet Res. 2018;20(6):e199.
Gitlin M. Lithium side effects and toxicity: prevalence and management strategies. Int J Bipolar Disord. 2016;4(1):27.
Grigsby-Toussaint DS, Shin JC, Reeves DM, Beattie A, Auguste E, Jean-Louis G. Sleep apps and behavioral constructs: a content analysis. Prev Med Rep. 2017;6:126–9.
Grundy QH, Wang Z, Bero LA. Challenges in assessing mobile health app quality: a systematic review of prevalent and innovative methods. Am J Prev Med. 2016;51(6):1051–9.
Henson P, David G, Albright K, Torous J. Deriving a practical framework for the evaluation of health apps. Lancet Digit Health. 2019;1:e52–4.
Hidalgo-Mazzei D, Mateu A, Reinares M, Murru A, Del Mar BC, Varo C, et al. Psychoeducation in bipolar disorder with a SIMPLe smartphone application: feasibility, acceptability and satisfaction. J Affect Disord. 2016;200:58–66.
Hidalgo-Mazzei D, Nikolova VL, Kitchen S, Young AH. Internet-connected devices ownership, use and interests in bipolar disorder: from desktop to mobile mental health. Digital Psychiatry. 2019;2(1):1–7.
Hidalgo-Mazzei D, Llach C, Vieta E. mHealth in affective disorders: Hype or hope? A focused narrative review. Int Clin Psychopharmacol. 2020;35(2):61–8.
Hoffman L, Wisniewski H, Hays R, Henson P, Vaidyam A, Hendel V, et al. Digital Opportunities for Outcomes in Recovery Services (DOORS): a pragmatic hands-on group approach toward increasing digital health and smartphone competencies, autonomy, relatedness, and alliance for those with serious mental illness. J Psychiatr Pract. 2020;26(2):80–8.
Huckvale K, Torous J, Larsen M. Assessment of the data sharing and privacy practices of smartphone apps for depression and smoking cessation. JAMA Netw Open. 2019;2:e192542.
Jake-Schoffman DE, Silfee VJ, Waring ME, Boudreaux ED, Sadasivam RS, Mullen SP, et al. Methods for evaluating the content, usability, and efficacy of commercial mobile health apps. 2017;5(12):e190.
Kendall T, Morriss R, Mayo-Wilson E, Meyer TD, Jones SH, Oud M, et al. NICE guidance on psychological treatments for bipolar disorder. Lancet Psychiatry. 2016;3(4):317–20.
Krebs P, Duncan DT. Health app use among US mobile phone owners: a national survey. 2015;3(4):e101.
Kupfer DJ, Frank E, Grochocinski VJ, Cluss PA, Houck PR, Stapf DA. Demographic and clinical characteristics of individuals in a bipolar disorder case registry. J Clin Psychiatry. 2002;63(2):120–5.
Lagan S, Ramakrishnan A, Lamont E, Ramakrishnan A, Frye M, Torous J. Digital health developments and drawbacks: a review and analysis of top-returned apps for bipolar disorder. Int J Bipolar Disord. 2020a;8(1):39.
Lagan S, Aquino P, Emerson MR, Fortuna K, Walker R, Torous J. Actionable health app evaluation: translating expert frameworks into objective metrics. NPJ Digit Med. 2020b;3(1):100.
Lagan S, Sandler L, Torous J. Evaluating evaluation frameworks: a scoping review of frameworks for assessing health apps. BMJ Open. 2021;11(3):e047001.
Lagan S, Camacho E, Torous J. Is there a clinically relevant, publicly accessible app for that? Exploring the clinical relevance and availability of mobile apps for schizophrenia and psychosis. Schizophr Res. 2021a;230:98–9.
Lagan S, D’Mello R, Vaidyam A, Bilden R, Torous J. Assessing mental health apps marketplaces with objective metrics from 29,190 data points from 278 apps. Acta Psychiatr Scand. 2021b;144(2):201–10.
Larsen ME, Huckvale K, Nicholas J, Torous J, Birrell L, Li E, et al. Using science to sell apps: evaluation of mental health app store quality claims. NPJ Digit Med. 2019;2(1):18.
Larsen M, Nicholas J, Han J, Lemon C, Okun K, Torok M, et al. Here’s something I prepared earlier: a review of the time to publication of cross-sectional reviews of smartphone health apps. BMJ Open. 2020;10(12):e039817.
Leitan ND, Michalak EE, Berk L, Berk M, Murray G. Optimizing delivery of recovery-oriented online self-management strategies for bipolar disorder: a review. Bipolar Disord. 2015;17(2):115–27.
Malhi GS, Bell E, Boyce P, Bassett D, Berk M, Bryant R, et al. The 2020 Royal Australian and New Zealand College of psychiatrists clinical practice guidelines for mood disorders: Bipolar disorder summary. Bipolar Disord. 2020;22(8):805–21.
Mathews SC, McShea MJ, Hanley CL, Ravitz A, Labrique AB, Cohen AB. Digital health: a path to validation. NPJ Digit Med. 2019;2(1):38.
Matthews M, Abdullah S, Murnane E, Voida S, Choudhury T, Gay G, et al. Development and evaluation of a smartphone-based measure of social rhythms for bipolar disorder. Assessment. 2016;23(4):472–83.
McHugh ML. Interrater reliability: the kappa statistic. Biochem Med. 2012;22(3):276–82.
Merikangas KR, Jin R, He JP, Kessler RC, Lee S, Sampson NA, et al. Prevalence and correlates of bipolar spectrum disorder in the world mental health survey initiative. Arch Gen Psychiatry. 2011;68(3):241–51.
Michalak EE, Murray G. Development of the QoL.BD: a disorder-specific scale to assess quality of life in bipolar disorder. Bipolar Disord. 2010;12(7):727–40.
Michalak EE, Suto MJ, Barnes SJ, Hou S, Lapsley S, Scott M, et al. Effective self-management strategies for bipolar disorder: a community-engaged Delphi consensus consultation study. J Affect Disord. 2016;206:77–86.
Miklowitz DJ, Scott J. Psychosocial treatments for bipolar disorder: cost-effectiveness, mediating mechanisms, and future directions. Bipolar Disord. 2009;11(Suppl 2):110–22.
Mistler LA, Ben-Zeev D, Carpenter-Song E, Brunette MF, Friedman MJ. Mobile mindfulness intervention on an acute psychiatric unit: feasibility and acceptability study. JMIR Ment Health. 2017;4(3):e34.
Morton E, Murray G. Assessment and treatment of sleep problems in bipolar disorder—a guide for psychologists and clinically focused review. Clinical Psychol Psychother. 2020;27(3):364–77.
Morton E, Nicholas J, Lapadat L, O’Brien HL, Barnes SJ, Poh C, et al. Use of smartphone apps in bipolar disorder: an international web-based survey of feature preferences and privacy concerns. J Affect Disord. 2021a;295:1102–9.
Morton E, Torous J, Murray G, Michalak EE. Using apps for bipolar disorder—an online survey of healthcare provider perspectives and practices. J Psychiatr Res. 2021b;137:22–8.
Morton E, Ho K, Barnes SJ, Michalak E. Digital health literacy in bipolar disorder: An international web based survey. JMIR Ment Health. In Press.
Muñoz AO, Camacho E, Torous J. Marketplace and literature review of Spanish language mental health apps. Digit Health. 2021;3(7).
Murnane EL, Cosley D, Chang P, Guha S, Frank E, Gay G, et al. Self-monitoring practices, attitudes, and needs of individuals with bipolar disorder: Implications for the design of technologies to manage mental health. J Am Med Inform Assoc. 2016;23(3):477–84.
Murray G, Suto M, Hole R, Hale S, Amari E, Michalak EE. Self-management strategies used by ‘high functioning’ individuals with bipolar disorder: from research to clinical practice. Clin Psychol Psychother. 2011;18(2):95–109.
Nicholas J, Larsen ME, Proudfoot J, Christensen H. Mobile apps for bipolar disorder: a systematic review of features and content Quality. J Med Int Res. 2015;17(8):e198.
Nicholas J, Fogarty AS, Boydell K, Christensen H. The reviews are in: a qualitative content analysis of consumer perspectives on apps for bipolar disorder. J Med Internet Res. 2017a;19(4):e105.
Nicholas J, Boydell K, Christensen H. Beyond symptom monitoring: Consumer needs for bipolar disorder self-management using smartphones. Eur Psychiatry. 2017b;44:210–6.
Nicholas J, Shilton K, Schueller SM, Gray EL, Kwasny MJ, Mohr DC. The role of data type and recipient in individuals’ perspectives on sharing passively collected smartphone data for mental health: Cross-sectional questionnaire study. J Med Internet Res. 2019;21(4):e12578.
O’Brien HL, Morton E, Kampen A, Barnes SJ, Michalak EE. Beyond clicks and downloads: a call for a more comprehensive approach to measuring mobile-health app engagement. Bjpsych Open. 2020;6(5):e86.
Patoz M-C, Hidalgo-Mazzei D, Pereira B, Blanc O, de Chazeron I, Murru A, et al. Patients’ adherence to smartphone apps in the management of bipolar disorder: a systematic review. Int J Bipolar Disord. 2021;9(1):19.
Qu C, Sas C, Daudén Roquet C, Doherty G. Functionality of top-rated mobile apps for depression: systematic search and evaluation. JMIR Ment Health. 2020;7(1):e15321.
Robillard JM, Feng TL, Sporn AB, Lai J-A, Lo C, Ta M, et al. Availability, readability, and content of privacy policies and terms of agreements of mental health apps. Internet Interv. 2019;17:100243.
Rubanovich CK, Mohr DC, Schueller S. Health app use among individuals with symptoms of depression and anxiety: a survey study with thematic coding. 2017;4(2):e22.
Schueller SM, Neary M, O’Loughlin K, Adkins EC. Discovery of and interest in health apps among those with mental health needs: Survey and focus group study. J Med Internet Res. 2018;20(6):e10141.
Sorkin DH, Janio EA, Eikey EV, Schneider M, Davis K, Schueller SM, et al. Rise in use of digital mental health tools and technologies in the United States during the COVID-19 pandemic: survey study. J Med Internet Res. 2021;23(4):e26994.
Tena-Cucala R, Cobo J. Usabilidad, satisfacción y facilidad de uso de aplicaciones (apps) gratuitas en español para pacientes con Trastorno Bipolar/Usability, satisfaction and ease of use of free apps in Spanish for patients with Bipolar Disorder. Psicosomática y Psiquiatría. 2019;2019:13–38.
Torous J, Brady R. Advancing care for bipolar disorder today and breakthroughs in access and treatments tomorrow with mobile health and smartphone apps. Bipolar Disord. 2020;22(3):211–2.
Torous J, Firth J, Huckvale K, Larsen ME, Cosco TD, Carney R, et al. The emerging imperative for a consensus approach toward the rating and clinical recommendation of mental health apps. J Nerv Ment Dis. 2018;206(8):662–6.
Urošević S, Abramson LY, Harmon-Jones E, Alloy LB. Dysregulation of the behavioral approach system (BAS) in bipolar spectrum disorders: review of theory and evidence. Clin Psychol Rev. 2008;28(7):1188–205.
van Bendegem MA, van den Heuvel SC, Kramer LJ, Goossens PJ. Attitudes of patients with bipolar disorder toward the Life Chart Methodology: a phenomenological study. J Am Psychiatr Nurses Assoc. 2014;20(6):376–85.
Wasil AR, Venturo-Conerly KE, Shingleton RM, Weisz JR. A review of popular smartphone apps for depression and anxiety: assessing the inclusion of evidence-based content. Behav Res Ther. 2019;123:103498.
Wisniewski H, Liu G, Henson P, Vaidyam A, Hajratalli N, Onnela J-P, et al. Understanding the quality, effectiveness and attributes of top-rated smartphone health apps. Evid Based Ment Health. 2019;22:4–9.
Woodward K, Kanjo E, Brown D, McGinnity TM, Inkster B, MacIntyre D, et al. Beyond mobile apps: a survey of technologies for mental well-being. IEEE Trans Affect Comput. 2020:1.
Yatham LN, Kennedy SH, Parikh SV, Schaffer A, Bond DJ, Frey BN, et al. Canadian Network for Mood and Anxiety Treatments (CANMAT) and International Society for Bipolar Disorders (ISBD) 2018 guidelines for the management of patients with bipolar disorder. Bipolar Disord. 2018;20(2):97–170.
The authors gratefully acknowledge the research participants who were involved in this project, the expertise of the CREST.BD Community Advisory Group and CREST.BD network members. EM was supported by a Canadian Institutes of Health Research Banting Postdoctoral Fellowship.
This research was supported by a Canadian Institutes of Health Research Project Grant: “Bipolar Bridges: a digital health innovation targeting quality of life in bipolar disorder”.
Ethical approval and consent to participate
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. All participants provided written, informed consent.
Consent for publication
EM, JN, LY, LL, SJB, MDP, CD, MC, and RK declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. EEM has received funding to support patient education initiatives from Otsuka.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Survey of App Use: Perspectives from people living with bipolar disorder.
Evaluation of the most commonly nominated mood and sleep self-management apps (n = 9) using the MIND framework.
List of studies evaluating the feasibility of the most commonly nominated mood and sleep self-management apps (n = 9).
List of studies evaluating the efficacy or validity of the most commonly nominated mood and sleep self-management apps (n = 9).
About this article
Cite this article
Morton, E., Nicholas, J., Yang, L. et al. Evaluating the quality, safety, and functionality of commonly used smartphone apps for bipolar disorder mood and sleep self-management. Int J Bipolar Disord 10, 10 (2022). https://doi.org/10.1186/s40345-022-00256-6
- Bipolar disorder