Updated: August 1, 2020 8:06:18 am
Should you get tested just to check your status?
It is not advisable – because the probability of being wrongly identified as Covid-19 positive is significant. When no test for the virus is 100% accurate, the key aspect to consider is the prevalence of the disease in the population that is being tested
If I test positive, do I have the disease?
The unequivocal answer to the question is: Maybe.
This uncertainty exists because few diagnostic tests are 100% accurate. The inaccuracy coupled with disease prevalence makes a positive result open to interpretation, and not a definite judgment on the health status.
The same vagueness exists even with a negative result; it need not be a certificate of good health. This article explains the underlying reasons, and provides some best practices for undergoing testing (applicable to Covid-19 and other tests) to increase confidence in the eventual diagnosis.
The accuracy of a diagnostic test has two components. This is because a test needs to accomplish two goals: identify the diseased and healthy individuals correctly. These two objectives are distinct, i.e., one can be achieved without the other. As an extreme example, imagine a test that labeled anyone undergoing the test as diseased. This test would perform remarkably if most of the individuals going for the test were sick, but it would fail miserably if the majority of test takers were healthy people.
The first measure is the test’s ability to correctly identify diseased individuals. This is called Sensitivity. A sensitive test picks up a large percentage of the disease. Following the common convention that a positive result indicates disease and a negative identifies healthy individuals, Sensitivity is calculated as the number of positives out of the number of diseased individuals tested.
The second measure, called Specificity, is the ability to correctly identify healthy individuals. Specificity equals the number of negatives out of the number of healthy individuals tested. A specific test is able to weed out everything that is “not the disease”.
False results: positives and negatives
Any false result – positive or negative – is a problem.
A false positive – that is, labeling a healthy person as diseased — leads to unnecessary emotional, financial, and health burdens due to additional testing and potentially wasteful and dangerous procedures. In the Covid-19 decisionmaking framework, a false positive places burden on the individual, their social network, and the government. Another cloaked danger is that the individual after getting “cured” — and in absence of any antibody tests – falsely believes that he or she is immune to the disease.
A false negative – indicating a diseased individual as healthy – creates a false sense of security and, depending upon the condition being tested, can lead to fatal consequences. In a Covid-19 setting, it leads to increased disease spread by the incorrectly cleared individual.
False results are a reality for most diagnostic tests. This reality means that the test result does not provide a guaranteed health status but rather, a likelihood (a probability) of being sick or healthy. The key to reaching a “correct” conclusion is to understand the mathematics linking the test accuracy and these probabilities.
Probability of false positives and false negatives
In addition to the Sensitivity and Specificity of the test, the number of false results also depends on the disease prevalence (per cent of population that is affected). Suppose an individual is being tested for an extremely rare disease. If the test returns positive, the likelihood of that person actually having the disease could be very low (depending upon the Sensitivity and Specificity of the test), i.e., the result is a false positive. In contrast, if a test for a highly common ailment comes back negative, then that negative result has to be questioned.
This link between the “belief” in the test result and the prevalence of the disease might appear confusing. The analysis shown below offers some guidelines for assessing these situations.
Consider a hypothetical test with 90% Sensitivity and 95% specificity. The table below shows the possible outcomes for 1,000 individuals tested. The above definitions of Sensitivity and Specificity are used to generate these numbers.
The main points are:
The disease prevalence is 5%, and that means 50 out of 1,000 are sick. The table shows that with the above test, however, 92 will test positive. Forty-seven are false positives (“False result” column).
The crucial math is the following: out of the 92 testing positive, only 45 are actually sick (cell A) – the remaining 47 are false positives. This means that the probability of being sick, if the test is positive, is only 49% (45/92). This is called positive predictive value (or per cent), or post-test probability of disease, and indicates the degree to which a positive test result can predict actual sickness (Cell E). This number should be as high as possible to drive confidence in the positive test result.
Similarly, there is negative predictive value (Cell F). This indicates confidence in a negative result. In this example, a negative result has a 99.4% probability of being accurate.
The reason for such high false positives becomes clear after studying the details of the calculation. Out of the 50 that are diseased, 45 (Cell A) are detected as sick (using the 90% Sensitivity of the test). Out of the 950 healthy individuals, only 903 (Cell D) test as healthy (using the 95% specificity of the test). The remaining 47 (Cell C) are all false positives.
Did the test achieve anything? Pre-test and post-test probabilities
One way to interpret the result is to look at the probability of being sick before and after the test. Before the test this probability – called pre-test disease probability – is 5% (same as that within the overall population assuming no additional information such as symptoms). Since a decision regarding future course of action cannot be taken with such low probability, the test is used to improve decision-making. After a positive test result, in the above scenario, this probability – now termed post-test disease probability – increases to 49%; a significant increase over the pre-test situation, but likely still not enough to guide decision-making, especially if the future course has some expensive and risky tests and procedures.
The test improves probability of being healthy as well (in case of a negative result). Before the test, it is 95%. After the test, it increases to 99.4%. A negative result, then, leaves only a 0.6% chance that the person might be sick.
Thus, a test is a tool that increases our confidence in either diagnosis. And a good test is one that raises this confidence to high levels. In mathematical lingo, a good test has high positive predictive value and a high negative predictive value (Cells E and F in the table above).
How can false positives be reduced?
The 47 false positives result from 95% Specificity and the 950 healthy individuals. 95% in most contexts is a high enough accuracy. In this situation of low disease prevalence, however, a test of greater Specificity is needed. To push the false positives down to, let us say, 10, a test with 99% specificity is needed. (1% of 950 is 9.5, which is rounded to 10.) Total positives become 55 and the positive predictive per cent increases to 82% (45/55), i.e., a positive result has an 82% chance of being correct.
Increasing Specificity is one way to reduce the number of false positives. The other lever, mathematically, is to reduce the relative number of healthy people. In practice, a reduction can be achieved by testing in a community with wider disease spread (i.e., fewer healthy people). This doesn’t mean waiting for increased spread, but rather limiting the test to a population that is more likely to have the disease.
The most obvious way is to limit to a population with specific symptoms or with some other identifiable associated condition or risk factor.
The numbers for the same test performed on a subset of the population (let’s say 500 with 50 within as sick individuals) are shown below.
The highlights are:
- The number of false positives is down to 22 – a direct result of reducing the healthy population to 450. Consequently the positive predictive per cent (post-test probability of disease) goes up to 67%.
- Number of false negatives stays at 5. As a percentage, however, the false negatives inch up (5/433 versus 5/908 previously). As disease prevalence increases, false negatives increase if Sensitivity is low.
Covid-19 related testing
There are two types of tests used for Covid-19. The test to detect Covid-positive cases is based on the Reverse Transcription Polymerase Chain Reaction (RT-PCR) technology. The second test detects the antibodies to the virus. Generally, it is used in surveillance programs to establish prevalence of the disease (also known as seroprevalence). This article does not cover the antigen test that can also reveal active cases by detecting part of the virus.
The accuracy data for either test is not widely known. One organisation in Geneva – Foundation for Innovative New Diagnostics (FIND) – is working with the WHO to collect test kits from all over the world and score the kits on Sensitivity and Specificity. Evaluation of RT-PCR tests from 22 companies in a laboratory setting shows Sensitivity ranges between 90% and 100%, and Specificity between 95% and 100% (as of June 23, 2020). Laboratory testing results typically show higher numbers as compared to clinical testing results (samples from patients); sample collection methodology, among other factors, impacts these results.
Sensitivity and Specificity for antibody testing kits developed in India is 92.4% and 97.9% respectively. [Sapkal et al, Indian J Med Res. 2020; 151 (5): 444-449] The first seroprevalence study by the Indian Council of Medical Research (ICMR), presumably with these kits, determined Covid-19 prevalence to be 0.73% in mid-May. [Government of India, PIB Press Briefing, June 11, 2020. Available at https://www.youtube.com/watch?v=jY-OmCw3QDE%5D. False positives will be a challenge at such low prevalence. A calculation using the above test parameters indicates that at 1% prevalence, recorded positives will be 3%, and almost 70% will be false. Since the objective of seroprevalence is to determine the spread of the disease in the population, options such as limiting the size of the population are not really meaningful. Therefore, further analysis and interpretation of calculated prevalence are required.
Population based statistics are fine for policy making. From an individual perspective, how does one decide whether to take the test? When should one trust a positive PCR result?
Confidence in a positive result (i.e., a high post-test disease probability) requires a high Specificity test conducted in a setting that has a “high enough” pre-test disease probability. Unfortunately both these numbers might be unknown.
Pre-test probability increases when testing is limited to symptomatic individuals and to people with some risk of exposure (e.g., primary contacts of known infected individuals). An estimate of this number is possible from the daily test positive rate (number of positive tests per total tests carried out). In countries such as India and US with rising case numbers, it is between 5-10%; in regions with higher spread it could be between 20%-30% (analysis not shown).
The figure below shows the dependence of post-test disease probability on pre-test probability for three high Specificity values with a special focus on pre-test probability between 5%-20%.
The table shows the values obtained from the chart above. As it reveals, high Specificity and high pre-test probability are crucial for confidence in a positive test result. Even a small change in Specificity – from 97% to 99% – results in a significant change, especially at low pre-test probability, in the post-test disease probability and in our confidence in a positive result.
Based on the above discussion, there are two best practices for decision-making at the individual level. The first and most important: testing in asymptomatic individuals “just to check” is counter-productive since pre-test probability of disease is low for such individuals. There is an understandable urge to reduce the uncertainty by seeking the comfort of a test result. The prudent decision is to counter this impulse with the knowledge that testing will not provide certainty unless conducted in the correct context. Therefore, asymptomatic individuals should opt for a test if they have a solid reason to fear exposure. Second: in a scenario where testing is required, the individual, if it is possible, should ask for the test kit with the highest Specificity.
The antibody test, at least for now, is used for prevalence estimates and the test result does not impact the individual.
📣 Express Explained is now on Telegram. Click here to join our channel (@ieexplained) and stay updated with the latest
Some commonly misused tests
Similar challenges posed by other diagnostic tests are covered only briefly in this article. The best practice, however, is the same for any diagnostic test: testing should only be considered in situations that have a high pre-test probability of disease. General screening of any population should be avoided if disease prevalence is low.
Annual diagnostic test packages for asymptomatic individuals violate this principle.
Stress testing to diagnose coronary disease is an example of a test used incorrectly frequently. Coronary disease prevalence in the general population is about 5%. The Sensitivity and Specificity values for the treadmill stress test are around 70%. [Gillinov, M., & Nissen, S. (2012). Heart 411. Harmony] Such low numbers make any result highly unreliable when the test is conducted as a routine check-up in asymptomatic individuals.
Summary of arguments
Testing is unlikely to provide 100% certainty about the condition being diagnosed. To have confidence in a positive test result, testing should be performed in a situation with elevated risk (i.e., a pre-test disease probability higher than the general disease prevalence) using a test with high Specificity.
Similar judgment is needed for confidence in a negative result as well. There is a benefit to performing these calculations prior to the test, since all these parameters are known (or can be estimated), to understand the implications of both possible results.
A true positive would be the involvement either of the MD or the diagnostic lab in making this determination with the individual.
Some additional notes
False positives have been a main focus of this article. Similar discussion on false negatives and post-test probability of health based on a negative result is possible.
Two related topics have not been covered in detail here. One is determination of Sensitivity, Specificity, and “actual” number of diseased and healthy individuals. The second is understanding of confidence intervals around the average values of test parameters. Interested readers can refer to widely accessible information on these.
The word “confidence” is used in the sense of “certainty” or “belief”, and not in the sense of a confidence interval.
Dr Tushar Gore’s focus area is pharmaceuticals. He studied at IIT-Bombay and the University of Minnesota, and has worked at McKinsey and Novo Nordisk. He is currently MD/CEO at Resonance Laboratories, a niche pharmaceuticals manufacturer.
📣 The Indian Express is now on Telegram. Click here to join our channel (@indianexpress) and stay updated with the latest headlines
© The Indian Express (P) Ltd