Some news reports have suggested that smartwatches or other wearable devices can detect signs of COVID-19 infection before it becomes symptomatic. If true, this could be a game-changer for managing the pandemic: People who know they are infected could begin to quarantine even before they develop symptoms, and thereby reduce community spread of the virus. Positive findings have been reported with multiple devices, including the Apple Watch, Fitbit (also here) or Garmin wrist trackers, and the Oura Health smart ring.
Primarily these studies have looked for changes in people's heart rate, although some of the analyses also include activity data and sleep patterns. A noted challenge of this and other chronic-disease-detection studies is that heart rate, activity, and sleep can be quite variable within individuals over time, even under normal conditions. Think about the last time you took a trip on an airplane across time zones, for instance -- a certain level of disruption of your usual sleep and exercise patterns is normal, and was likely associated with changes in your heart rate pattern as well. And in my own research, heart rate variability served as a pretty good marker of psychological stress, which we are all experiencing at higher than normal levels during the pandemic. Like temperature (which some of these devices also measure), heart rate is therefore a nonspecific marker — it can suggest that something is different, but can’t necessarily tell you that the something in question is infection with COVID-19. One of the studies with data from multiple device types used resting heart rate as a predictor instead of heart rate variability; this seems more promising as resting heart rate is generally a sign of heart health and might be particularly sensitive to the heart-damaging effects of COVID-19.
A newer wrist-worn device called WHOOP is designed to detect respiratory rate changes (e.g., during sleep), and recently has been studied as a way to identify pre-symptomatic COVID-19 infection. Breathing is more clearly related to the severe complications of COVID-19 that can land people in the hospital, and therefore seems like a good candidate for predictive modeling. But a problem with this indicator is that most currently available sensors can only estimate breathing instead of actually measuring it. That seems to be the case for the WHOOP device as well. Generally wrist-worn devices use optical sensors that estimate oxygenation in the blood -- the same way that a pulse ox monitor works on your finger. (Another device, the Spire medical tag, uses pressure sensors to measure respiratory effort, but doesn't seem to have been evaluated yet for COVID-19 detection).
As exciting as these results are, they come with some important caveats. First, much of the research has been sponsored (as you might expect) by companies with a financial interest in the technology. That's not necessarily a bad thing -- in fact, corporate-sponsored research often has the resources behind it to allow for a more full and careful evaluation of results -- but it does mean that we should exercise additional caution before jumping on the bandwagon. Ideally one would like to see independent replication of findings by different teams of investigators, and that type of evidence is starting to be available for the heart rate studies that relied on data from several different types of devices. Most of these articles have not yet completed the peer-review process, so additional gaps in their research design or interpretation might still be noted by the scientific community, but they do seem at least suggestive of an effect.
Assuming that the findings are in fact valid, what do they mean? Some of the studies showed only that a metric like heart rate variability (HRV) differed significantly between people who did versus did not have COVID-19. That’s a weak finding because it might not mean heart rate is an early sign of illness; it might instead just mean that people with low HRV are more vulnerable to more severe COVID-19 illness. This seems likely, because we know that pre-existing heart problems are a major risk factor for COVID-related mortality. An additional weakness of all the studies is that they didn’t systematically test everyone for COVID-19, so some of the “COVID-free” participants might actually have had the virus in asymptomatic form. That would be particularly likely if low HRV was actually a risk factor for symptom severity rather than an early warning sign of illness.
A better way to evaluate diagnostic tests is based on two characteristics called positive predictive value (PPV) and negative predictive value (NPV). PPV is the percentage of the time that the test is right when it says that you do have some condition, and NPV is the percentage of the time that it's right when it says that you don't. The two measures can be quite different, leading to different problems of interpretation. For instance, a test with a PPV of 90% but an NPV of only 50% is pretty useful if it says you're infected -- you probably are. If it says on the other hand that you aren't infected, there's still a 50/50 chance that you are. (For more about the differences that can arise in interpreting positive versus negative test results, see this article with several user-friendly examples). Given the high asymptomatic rate of COVID-19 (the latest population-level estimates suggest that more than half of people who are infected and can transmit the virus to others will never go on to develop symptoms), NPV is the statistic of most interest for sensor studies. Findings about the PPV might be useful if we were willing to quarantine on the slightest suspicion that we were infected. But the behavioral evidence so far suggests that isn’t true. What we really want is a test to tell us that we aren’t currently carriers of COVID-19, so that we can go about living our lives. The current research unfortunately doesn’t speak to that question at all.
What the sensor studies tend to report is their devices' sensitivity, which is a measure of what percentage of known cases were successfully detected in advance, and is more strongly related to PPV than to NPV. (For more about the difference between sensitivity/specificity and PPV/NPV metrics, see the tables in this article. For our purposes here, it's enough to note that sensitivity is less useful clinically than PPV, because it starts from the assumption that we know the truth and want to determine the accuracy of the test, when in clinical practice all we usually have is the test result and we want to use that to infer the truth). Ambulatory sensor data from the Oura smart ring were only slightly better than chance at detecting fever, and fever (which we already screen for in many contexts) is only slightly better than chance for identifying people infected with COVID-19. In the FitBit studies (here and here), sensor data had additive effects to typical symptom screening, improving its overall accuracy from 71%-78% to 80%-82% (Area Under the Curve, which reflects both sensitivity and specificity). That’s good, but we could have gotten most of the way there with just a typical symptom screening, and additionally we don’t know if we could have picked up any of the asymptomatic-but-infectious people. In one of the best studies that looked at multiple sensor measures across multiple devices, overall ability to detect COVID-19 (confirmed by a positive test) was 63%, which is slightly better than chance but not a lot better. That result also again ignores anyone who did have COVID-19 but never knew it because they had no symptoms and weren’t tested. It’s likely that those folks represent only about half of total cases.
At the moment, my reading is that the evidence supports using wearables to achieve a slightly augmented detection rate, maybe a day or two sooner, for those people who will eventually go on to show symptoms. Without taking into account the false positive rate, there might be a cost in terms of quarantining more people who don’t in fact have COVID-19 but have worse heart rate or breathing results for some other reason. And because we’re most interested in NPV with a frequently-asymptomatic condition, the studies so far unfortunately can’t tell us what we most want to know. Some sensor measures may yet prove to be helpful in ruling out COVID-19 infection, with resting heart rate and respiratory rate being probably the most likely candidates. But to prove that, we need studies that gather sensor data on a large number of people and then systematically test all of them regardless of symptoms. The prospective design used in most of the current studies is a good one, but it would be better with frequent COVID-19 testing of all participants rather than just waiting to see who comes down with symptomatic disease. That type of design is what’s necessary tell us what we most want to know, the NPV of readily available sensor data to tell us whether it’s safe to go about our lives while COVID-19 is still circulating in our community.
Comments
Post a Comment