.svg){kind=icon}
R&Q's April 2020 webinar focused on Structuring Performance Evaluation Reports (PERs) under In Vitro Diagnostic Regulation (IVDR). In this blog post, our knowledgeable IVDR experts sat down to provide answers to your questions from the webinar and we hope that they will shed some light on similar questions that you may have on the topic.
We all know that the clock is ticking down to the IVDR date of application! May 26, 2022 sounds far away, but it's really not considering the size of the step to IVDR compliance for most IVD companies. The transition from MDD to MDR for medical device manufacturers feels relatively small in comparison to the 85% of IVD companies moving from self-certified to notified body audits, complex regulatory systems, and the expectations of much tighter controls over the web of virtual manufacturing and distribution relationships that IVD companies are accustomed to.
R&Q is here to help address any other questions that you may have regarding audits or any other IVDR related topics. Contact us today to find out how we can help! Have more questions on IVDR? Join us on Thursday, June 18 for the discussion during RQM+ Live! #10 Transition to IVDR: How far along should we be? Click here to learn more and register for free.
If you were not able to attend this webinar, here's a link where you can sign up to view the on-demand version (and download the slides).
Q: Should we include Post-Market Surveillance (PMS) data of similar products and our device only or do we need to include all results for the product code/GMDN term searched?
A: To evaluate your product’s safety and performance in the PER, it is only necessary to address the data on your device.
While not needed for the PER, data on other products in the field can help identify any new risks that should be considered about your product, as part of your risk management process.
Q: What happens if in creating this report we find that there is a potential problem with our IVD (e.g. lab variations) what do I have to do?
A: With this situation, you may need to go back and assess your analytical performance. You would need to ensure that all the criteria around specificity, sensitivity, etc., were all captured properly. If you are now seeing field data that is showing that there is a variation (as long as your analytic performance seems sound), you may need to look at any new risks that came out of that data finding. There may even be a potential need to go back and repeat analytical performance to confirm what the actual condition is.
Q: Where else is the State of the Art (SOTA) referenced/used? What are some good strategies to write this section when there are no similar products on the market, or our product has a different operation than the most-similar product on the market?
A: Information from your SOTA analysis is also used in post-market documentation. Literature and information on similar devices are summarized, to some extent, in the post-market performance report and post-market surveillance report. You would want to have that summary information and leverage the information you collected in the SOTA, about similar devices and alternative therapies, included in your post-market reports.
In some cases, when there is not a lot of literature on other devices, you can go and look at the databases. If your device has a different operation than most similar products on the market, you may look at alternative treatments. The overall idea of the SOTA is to provide a baseline for your device so that you can provide a thorough benefit-risk analysis and profile. You have your device benefits and risks and have identified what it can do that other devices cannot do. You then compare that against currently available treatments and use those as a baseline to show if the benefit-risk profile is acceptable.
Q: How should the PER be organized/products grouped when 1) a single technical file covers multiple, related products or 2) similar products each have their own technical file?
A: In general, you can structure them however you want. It just needs to be clear to the Notified Body reviewer what data applies to what product. An issue that has been seen, when manufacturers combine numerous products together, is that the products aren’t that similar (ex. have different intended uses). These should not be combined. Products should only be combined if they are truly similar, such as being different types of models. However, if you do choose to combine products, consider that it can create a long and possibly confusing report. You must make it clear what data is available for each device and how the general safety and performance requirements are supported for each device. The goal is for clarity, to ensure easier review by the Notified Body. We have seen instances of manufacturers that combined too much, and the Notified Body requested to have the products separated out.
Q: What is a good strategy for finalizing documents before submission, i.e., how much time can lag behind data in the PER and the submission for review?
A: We have seen, on the device side, that if your data gets past six months and up to one year, the Notified Bodies are not as comfortable with that. The data in the PER and submission should be fairly closely aligned in terms of timing and time frame. We would suggest that the data should be less than a year, but preferably within six months.
Q: Is it better to do a great job to start or will that set NB expectations too high and we should instead focus on the bare minimum to meet the regulations?
A: If you meet the requirements at the lower end, you are still technically in compliance. To be compliant though, even at a minimum, you need to ensure you have well-justified arguments to support your intended use of your device and provide all the appropriate supporting evidence and reports for review. You need to meet the regulations and need to provide evidence that you meet the applicable general safety and performance requirements.
As the Notified Bodies start looking at different products and different companies, they will start to set their baselines. Often, you don’t set a
Notified Body’s expectation, but the Notified Body sets their expectations for you. Sometimes it is a balancing act to figure out what they want, especially with all the new regulations and requirements.
It would be suggested to aim higher than the bare minimum and be as robust as possible. Consider the different products covered under your certifications and set the bar a good way above the minimum to demonstrate to the Notified Body that you have a good understanding of the regulations and how to meet the requirements of them.
To hear more around this question, join us on June 18 for the discussion during RQM+ Live! #10 Transition to IVDR: How far along should we be? Click here to learn more and register for free.
Q: We have been struggling with our first PER, what advice do you have for someone that has never done one?
A: You definitely would want to work with someone who has worked, at the very least, with clinical evaluations and would be familiar with the latest regarding what the Notified Bodies are expecting. CERs are much further along, as far as understanding what the Notified Bodies expectations are. We would also encourage you to review the Global Harmonization Task Force (GHTF) and MEDDEV guidance documents that we referenced in this webinar:
• GHTF/SG5/N6:2012 Clinical Evidence for IVD Medical Devices – Key Definitions and Concepts (PDF)
• GHTF/SG5/N7:2012 Clinical Evidence for IVD Medical Devices – Scientific Validity Determination and Performance Evaluation (PDF)
• GHTF/SG5/N8:2012 Clinical Performance Studies for In Vitro Diagnostic Medical Devices (PDF)
• MEDDEV 2.7/1 Rev 4 Clinical Evaluation for Medical Devices (PDF)
Q: Our literature search criteria returned >500 results. How do I limit the criteria to get a manageable amount of literature?
A: You should have inclusion/exclusion and appraisal criteria well established. When in a situation that you have a large number of results, and they all look somewhat suitable and could help your report, you can reduce the resulting amount by looking at sample sizes in the studies or you can reduce the time frame that you are looking at. You also need to start to look at the quality of data and the level of evidence in terms of if it was a controlled trial. Remember, sufficient evidence means more than just the number of patients or studies, but also the quality of the evidence. You want to select those with the highest quality of evidence.
Additionally, you don’t want to introduce bias and just select the data that have favorable results for your product. You need to ensure your exclusions are non-biased to allow inclusion for favorable and non-favorable results if they are relevant.
Q: We have the product labeled for one thing, but when we did our literature search it seems like doctors are ordering the test for another purpose, what do I have to do?
A: It depends on if this is a one-off situation or if it is systematic off label use. If it seems to be systematic, then you need to consider if the intended purpose of your product is correct and if any modifications should be made. If your intended purpose is correct, then you should establish some risk mitigation measures to ensure doctors are not using it primarily for that other purpose that you are not labeled for. This could include the revision of your warnings and cautions in your Instructions for Use (IFU) to make it clearer. These could become liability issues if the product is used off label and something adverse happens. It could still come back on the manufacturer if it were felt the IFU wasn’t clear enough in terms of off label use.
Q: How do you think the literature searches that are often performed by R&D could help/be usable for the literature searches required for clinical evaluation? There might be some overlap.
A: Literature searches should be conducted systematically and geared towards supporting the various elements of the PER. It is recommended that activities are coordinated across groups to ensure a consistent approach to performing the literature searches while keeping in mind the objectives for each group.
Q: We have a lot of the required content for PERs in multiple documents that are continually updated by our research team on a yearly basis. In the PER, can we reference those documents to relieve a lot of required updating in the future, or do we have to have all the information and data within the PER?
A: I think it is appropriate and practical to reference these documents along with a summary within the PER. It should be verified that the content and objectives of the external documents meet the requirements for the PER as defined in the regulation. It is also recommended that the data from external documents are representative of the current design of the products in scope.
Q: How can you assign the specific information of 9.1.(b) especially for our IVD product if you don't conduct a clinical performance study (following the rationale of GHTF guidance for established and standardized products)?
A: Even if a formal clinical performance study is not conducted, it is expected that data to support clinical performance is derived from published literature studies and supplemented with post-market data. The parameters as described in 9.1 (b) may be derived from the analytical performance data and then verified through clinical performance.
Q: Will you cover CDx and clinical trial assays?
A: This webinar did not cover companion diagnostics (CDx) or clinical trial assays.
Q: Do you have an opinion on ScienceDirect for full-text searches in place of Google Scholar?
A: If Science Direct allows for true full text searching along with formal indexing, then it may be more efficient than using Google Scholar.
Q: Will the slides be shared?
A: Yes! The slides and recording from this webinar can be found here.
Q: Is there a source available for definitions like 'analytical sensitivity' and 'analytical specificity'?
A: While both the IVDR and GHTF/SG5/N6:2012 explain analytical sensitivity and specificity, neither offers true definitions.
As described in a 1997 paper by Saah and Hoover, Analytical sensitivity "represents the smallest amount of substance in a sample that can accurately be measured by an assay."
"Analytical specificity" refers to the ability of an assay to measure on particular organism or substance, rather than others, in a sample." The full citation is Ann Intern Med. 1997 Jan 1;126(1):91-4. "Sensitivity" and "Specificity" Reconsidered: The Meaning of These Terms in Analytical and Diagnostic Settings. -Saah AJ1, Hoover DR.
Need additional assistance? Say hello to your expert MDR and IVDR implementers. Our team is ready to help with validated process, procedures, and tools. Click here to learn more about our EU MDR and IVDR services.
Sources:
1) International Medical Device Regulators Forum. Global Harmonization Task Force on Medical Devices (GHTF) Documents. Accessed 8 June 2020. http://www.imdrf.org/
2) Saah AJ, Hoover DR. "Sensitivity" and "specificity" reconsidered: the meaning of these terms in analytical and diagnostic settings. Ann Intern Med. 1997;126(1):91‐94. doi:10.7326/0003-4819-126-1-199701010-00026. Accessed 8 June 2020. https://pubmed.ncbi.nlm.nih.gov/8992938/
3) EUROPEAN COMMISSION, DG Internal Market, Industry, Entrepreneurship and SMEs. Consumer, Environmental and Health Technologies. Health technology and Cosmetics. MEDDEV 2.7/1 revision 4. GUIDELINES ON MEDICAL DEVICES. CLINICAL EVALUATION: A GUIDE FOR MANUFACTURERS AND NOTIFIED BODIES. Revised June 2016, Accessed 8 June 2020.
.svg "white RQM logo")
