The benefits of internet-based cognitive-behaviour therapy (iCBT) for depression and anxiety have been observed and replicated in many different settings, samples, and countries. The vast amount of research behind it has led iCBT interventions to be recommended by the National Institute for Health and Care Excellence as a treatment alternative for depression and anxiety.
Currently, iCBT is successfully used within stepped-care models for mental health care in several countries, such as the Netherlands and the UK, which offer iCBT solutions for individuals showing mild to moderate presentations of depression and anxiety. Furthermore, different organisations have set up specialised iCBT clinics in different countries where they offer mental health services for individuals with depression and anxiety (Titov et al., 2018).
As can be noted, the utilisation of iCBT as a treatment alternative is increasing and so is the research on how these interventions operate. In this vein, it is not hard to imagine that all the interactions the user have with the platform leave behind a track of analytic events that can be observed retrospectively. All this record of objective data has opened a whole avenue for research on metrics of usage of internet-delivered interventions and how this usage is related to outcomes.
Several reviews have explored the relationship between usage and outcomes across different studies (Donkin et al., 2011; Sieverink et al., 2017). These reviews have found that different studies report different metrics, with the number of modules completed and the number of logins being the most commonly reported metrics. Yet, the results of these studies are somewhat mixed and the specific contributions of these metrics to the outcomes remain unclear.
The concept of adherence in internet-based interventions
Although it may sound like a simple concept, in the last decade there has been a big debate in how adherence should be defined and measured. Historically, adherence has been reported in terms of attrition or dropout from a trial, that is, the number of participants who drop out or stop using the program before completion (Eysenbach, 2005). However, this definition provides limited insight into users’ interaction with internet-based interventions and how these interactions can be linked with outcomes.
Recent studies have reframed the concept of adherence in a way that it is more aligned to the intended use of the platform or “the extent to which individuals should experience the content to derive maximum benefit as implied by its creators” (Kelders et al., 2012). Following this definition, the intended use should be the minimum requirement for someone to be considered adherent to the intervention (Sieverink et al., 2017).
To determine the optimal dose of the intervention the developers have to consider the balance between the theoretical efficacious dose (i.e. exposure to the whole program) and the effective dose (i.e. real levels of exposure) that can be only determined upon application (Voils et al., 2014). Therefore, finding the most relevant metrics and determining thresholds of usage for these metrics are key to determining the optimal dosage.
Results of the study
The current study used data from 216 participants who completed pre and post measures within the context of a randomized controlled trial aimed to examine the efficacy of a supported iCBT intervention for depression (Space from Depression) over a period of 8 weeks (Richards et al., 2015). The primary outcome for this study was the Beck Depression Inventory (2nd edition, BDI-II), which is a validated and widely used measure for depression.
The sample was divided into those who obtained a reliable change in this measure and those who did not. We looked at different usage metrics such as time spent, number of logins, percentage viewed, number of reviews from the supporter, number of activities, number of messages from the user to the supporter and composite measures (i.e. time per session, activities per session). For each of these metrics, we compared the usage between the two groups. Only time spent, number of activities, number of logins and percentage viewed were significantly different between these groups. We further explored these metrics by comparing the usage weekly. Interestingly, the results showed that significant differences between groups could be observed from the first week onwards. The results also showed that the usage was higher during the first 4 weeks compared to further weeks.
We also did analysis to determine the optimal dose for each of these metrics. Our results showed that specific levels of platform usage, such as spending 7 hours, logging in 15 times, using 30 tools, and viewing 50% of the programme, were associated with a higher likelihood of obtaining reliable change.
Future implications
This study will serve as a first step to determine optimal levels of usage that can be used to establish thresholds for adherence in internet-based interventions. Determining these usage levels could inform the development of more responsive interventions, allowing for tailored, real-time recommendations based on usage levels to enhance the engagement of different types of participants. This would enhance the likelihood of a positive outcome and help digital interventions drive the field of mental health research forward. You can find the full publication of our study here.