Supporting patients in the selection of mental health Digital health tools amidst a pandemic: a demonstration project

When the pandemic struck, we figured that a very prevalent symptom of the COVID19 wouldn’t be anosmia. Instead, people would have to adapt and face an increase in their stress. We wanted to validate the effect of mental health apps in this setting.

Supporting patients in the selection of mental health Digital health tools amidst a pandemic: a demonstration project

At Therappx, we have curated and simplified Digital health tools’ (DHT) adoption since early 2018. During the last two years, we gathered an extensive collection of continuously updated data regarding the safety, therapeutic value and clinical relevance of hundreds of patient-facing tools (read more here on how exactly we do this).

When the pandemic struck, we figured that a very prevalent symptom of COVID-19 wouldn’t be anosmia. Instead, people would have to adapt and face increased stress. To support the general population, we decided to release to the general public, for free, a subpopulation of our dataset to assist patients in finding solutions to face this increase in stress.

On March 30th, after a full week of intense software development, we launched a platform to support patients in selecting DHT among thousands of options available in Apple’s App Store and Google’s Play Store. In this platform, patients can find, in just a few clicks, a validated, safe Digital Health Tool that corresponds to their personal needs and objectives.

Seventy-five days later, seeing the social impact we generated, we are confident that this was one of the best decisions we have made since we launched Therappx.


Available for free to any patients who visited covid19.therappx.com, we ended up helping 9'592 patients in 75 days who needed assistance in selecting a Digital Health Tool to reduce their stress levels.

Figure 1. Daily users of covid19.therappx.com, n = 9'592.

Of those 9'592 patients, 45,4% completed what we refer to as our onboarding process. They provided us with information (language preference, device availability, country of residence, etc.) to be matched by our proprietary algorithm with one of the carefully selected DHT.

For a better and safer match, patients could also provide us their baseline anxiety level (only if they wanted to). Eight hundred eighty-two (882) patients (or 9,2%) completed the initial assessment and authorized us to reach back in a week after visiting the platform to assess if the matched tool helped reduce their stress and anxiety.

Figure 2. Demographics of patients providing their anxiety level before being matched with a tool, n=882.

Real-World Evidence (RWE) generation as a way to support a stepped-care approach

From a clinical standpoint, we need to validate that self-care approaches lead to sufficient improvements, even if patients live with low-intensity or mild-to-moderate stress symptoms. For data-driven insights, the best way to assess such improvements is by asking patients to complete standardized, evidence-based questionnaires.

In anxiety, one commonly used questionnaire is the General Anxiety Disorder (GAD-7) scale. Psychologists and doctors use this questionnaire (or assessment) every day in practice. Patients may complete this assessment with no direct supervision.

Our objectives at Therappx were to use the GAD-7 to:

  1. Divert patients from solely using DHT and other self-help products if their anxiety levels are severe (GAD-7 ≥ 15);
  2. Ensure patients with mild-to-moderate anxiety levels (GAD-7 ≤ 14) improve their symptoms before or on the 7th day, as measured by a weekly questionnaire sent by email. At this time, we also verified that the patients were using their recommended DHT. If they didn’t reach lower anxiety thresholds or were not using the recommended tool, our app would offer to use other solutions supported by our proprietary algorithm. In-person care was also recommended for patients not achieving target values.
Figure 3. Repartition of GAD-7 results from patients (on a scale from 1 to 21), n=882.

We found that among patients who completed the anxiety questionnaire before being matched with a tool, 208 patients (23,6%) had a GAD-7 ≥ 15 (severe anxiety) and 204 patients (23,1%) completed at least one other reading of their anxiety level in the next four weeks.

Figure 4. Evolution of GAD-7 scores in a subpopulation of patients (April 2020), n = 58.

Of those who completed at least one assessment after being matched with a tool, 171 patients (83,8%) improved their anxiety levels by at least one point on the GAD-7 scale. Fifty-three (53) patients (28,9%) had their scores reduced sufficiently from severe anxiety thresholds to milder forms of anxiety.

Therefore, our platform demonstrated that it supports patients for a patient-centric and personalized selection of DHT. Moreover, it leads to better outcomes. Finally, it validates that a DHT leads to desirable outcomes for each patient (reducing stress, in this case). Therefore, it uses RWE to power stepped-care approaches in healthcare.

The next step for Therappx will be to expand this platform to another therapeutic field: mood problems. In a pilot project with Quebec’s Ministry of Health and the Ministry of Economy & Innovation, we will evaluate how we may leverage RWE to support our recommendation systems. We develop this technology supported by advisory services and research and development funding from the National Research Council of Canada’s IRAP program.


Secondary findings

1. Pointing to only one Digital Health Tool is too constraining for patients

As we focused on iterating the platform based on patient comments, we received crucial feedback on patients’ selection of DHT. Patients asked for other alternatives, as we initially pointed to only one DHT. We altered our algorithm to provide patients with best-fit recommendations and two options. It was well-received by subsequent users. Patients estimated that more than three recommendations might be too high to make a more straightforward selection.

In our next research, we’ll determine the absolute number of choices that we may offer to patients for simple but empowering decisions.

2. Our platform’s data could also be (secondarily) used by public health officials

We successfully gathered data that could significantly help public health officials, especially during the pandemic. For example, we may use data on populational anxiety levels to prioritize care provided to particular subgroups of patients (e.g. according to age group or specific areas).

Figure 5. Anxiety level as measured with GAD-7, according to age groups, n=882.

3. Our platform may help app manufacturers obtain data on patients’ objectives regarding using Digital Health Tools.

To help patients find the DHT that best fits their needs, we collected data on the desirability of features commonly found in mental health apps. For example, we now know that during the pandemic in Canada, patients with stress-related issues were more interested in a tool that contained an online community than a chatbot.

According to our research, this kind of data is not available for DHT manufacturers, even if it could significantly help prioritize developing new features, among other findings.

In summary

Therappx’s platform and app pairing algorithm demonstrated they are useful and efficient in enabling patients to make appropriate self-care decisions regarding Digital health apps.