What is the best antibody test for COVID-19?
Author: Dr Mark Ali MBBS BSc FRCS CTh, Medical Director
There is an overwhelming demand for an antibody test for Covid-19 and companies globally are racing to produce them. This coronavirus test tracker displays information on all of the commercially available coronavirus tests. As of May 14th, there are 179 available!
It is a huge but really important task to sift through all of these different offerings to find those that are the best. But what do we actually mean by “the best”?
What do we mean by “best”?
What constitutes the “best” test will mean different things to different people depending on what they are optimising for (mode of administration, false positive rate, false negative rate, turnaround time etc.). Different features aside, the “best” test will be the test that is fit for the intended purpose.
This may mean that different Covid-19 tests are required depending on the scenario in which they are going to be used. For example, an antibody test used to diagnose someone with current infection may have different requirements from those used to show if someone has immunity to Covid-19. The requirements for use at a population level may be different from those used individually where a clinician can review the clinical case in context, provide informed consent and explain the results with limitations in mind.
The Medicines & Healthcare Products Regulatory Agency (MHRA) provides guidance for the Target Product Profile (TPP) of a test: the TPP outlines the desired “profile” or characteristics of the target product that is aimed at a particular disease. It covers the intended use, target population and other desired attributes including safety and performance characteristics.
For antibody tests, the MHRA describes optimal characteristics for the tests and minimal criteria for defined features. They recognise that “a test that does not yet meet all these profiles may still have a role in supporting the UK testing strategy”.
Factors to consider:
1. How the test is the sample obtained?
a. Fingerprick to get capillary blood sample
b. Venous blood sample
c. Non-invasive methods e.g. saliva
2. How is the test analysed?
a. Is a laboratory required (with associated transport, requirement for skilled analysis, equipment, delay in results) or is the test self-contained?
3. Sensitivity: false negative rate
4. Specificity: false positive rate
For an explanation of sensitivity and specificity, please click here
An ideal test-the “best” test would be a non-invasive, point of care/near patient test that can unambiguously be interpreted there and then. It would have very high sensitivity and specificity. At the moment, this test does not exist! Therefore we have to optimise based on the intended use of the test.
A Covid-19 test that will be rolled out to the population at large will need to meet these stringent criteria. It would be logistically and economically very demanding to collect and send millions of blood samples to laboratories for analysis. People will need a definitive answer with no ambiguity as there will not be capacity to discuss tests on an individual basis. In particular, the tests must be highly specific- we cannot have people test positive for antibodies when these in fact represent an immune response to a cold they had back in January. This would provide false reassurance and the individual may go on to spread the virus unknowingly.
The requirements are arguably slightly different in private practice. We have the ability in private practice to offer laboratory-based testing and discuss the test results in clinical context and in the context of the limitations of the particular test.
Of course, we would want to offer the absolute best test meeting all of these criteria. And when it is available, we will be the first to offer.
How do we evaluate antibody tests?
Antibody tests are evaluated for their sensitivity (ability to detect all true positives i.e. low false negative rate) and specificity (ability to not falsely detect other antibodies as positive ie. false positive rate). Other considerations include: ease of use, invalid rate and operational characteristics.
When we study them, we look at the fine print of the manufacturer’s description together with all available independent studies of the test.
To evaluate the Covid-19 antibody test, it must be trialed against a reference standard. However, there is no currently agreed reference standard for establishing specimen immunity status. We know that it should be tested against samples from known positive cases and also known negative cases as well as cases of similar coronaviruses to ensure no cross-reactivity. Ideally, you want to trial as many samples as possible and you want the samples to represent a broad spectrum of features with different ages and severity of disease.
Here are some of the questions we ask when looking at the fine print of the testing specifications.
Who were the positive cases? How were the positive cases identified and how many samples have actually been tested? Many of the studies classify a known positive case as an individual who has had a positive swab result. But what you ideally want is to identify those who definitely have antibodies to covid-19 (the gold standard reference) and then trial the new antibody test against that. It has been reported that some individuals who have had covid-19 do not produce antibodies. If this is a true result, a negative test may in fact be accurate rather than a sensitivity issue. We need to be able to tell the difference and so the antibody test should be trialed against the gold standard rather than a proxy.
Tests for covid-19 initially focused mainly on hospitalised patients and so most of the samples represent moderate to severe disease. They are also likely to be from older individuals as we know these people are more likely to suffer more severe disease. It is likely that the antibody responses in these patients will be different from those with mild or asymptomatic disease. We do not know therefore if an antibody test that works for these individuals is generalisable to the population at large. It may be that the tests are fit for different purposes.
Ideally, we want the “n” number (the number evaluated) to be as high as possible. Some commercial tests available have been tested on very small numbers e.g. 25. In this case, claims of “100% sensitivity” or “100 specificity” should be viewed with caution.
The MHRA demand > 98% sensitivity based on a sample of 200 confirmed positive cases.
How many negative samples were tested? When were the negative samples obtained? Have the tests been evaluated against other similar coronaviruses to ensure no cross-reactivity?
We need to look at the numbers of known negative samples that were tested. When were the negative samples collected? For avoidance of
doubt, the sera should be from pre-covid-19 i.e. before December (or even November) 2019. Ideally, the manufacturers should have tested against other related coronaviruses to see if there is a cross-reaction.
The MHRA demand >98% specificity based on testing of at least 200 confirmed negative cases or from testing of specimens collected at least 6 months before the known appearance of the virus.
“CE marked”, “MHRA approved”, “FDA approved”: what does this mean?
A CE mark is a certification mark that is placed on medical devices to show they conform to the health, safety and environmental requirements in the directives. It applies to products sold within the European Economic Area but it is a worldwide recognisable standard. The CE mark shows that the device is fit for its intended stated purpose and meets legislation relating to safety.
A CE mark does not mean that a product is approved by the MHRA (Medicines and Healthcare products Regulatory Agency). The test must meet the standards outlined by the MHRA in their target product profile.
The US Food and Drugs Administration (FDA) is responsible for regulating medical devices such as antibody tests. The secretary of the Department of Health and Human Services granted emergency use authorisation (EUA) for medical devices to test for covid-19. This is to ensure that products can be accessed as quickly as possible. Commercial manufacturers must submit emergency authorisation requests along with validation data. The tests will then be independently evaluated by federal laboratories.
Which tests are currently thought to be the best?
Public Health England has approved the Covid-19 antibody test by manufacturer Roche. This test is performed on a blood sample that is analysed in the laboratory.
The Roche test is a Electrochemiluminescence immunoassay (ECLIA). Both looks for IgG antibodies.
According to Public Health England analysis:
Roche
Tested on 93 PCR positive patient samples, 50 confounder negative samples, 35 Porton negative samples, 387 Manchester negative samples
- All negative samples tested negative by the assay, giving a specificity of 100%. This accords with the manufacturer’s reported specificity of 100%.
- The assay gave an overall sensitivity of 83.87%, with a sensitivity ≥14 days of 87.0%.
- This is lower than the manufacturer’s reported sensitivity of 100% for samples taken ≥14 days post symptom onset. The sensitivity of the assay at ≥21 days post symptom onset is 87.7%, rising to 100% for samples >40 days after symptom onset