Want to know which pitfalls to avoid in your MedTech Clinical Evaluation? Look here
Helpful guidance addressing the clinical evaluation of medical devices is difficult to come by. This is because, unlike, for example, the Regulatory space, there is a dearth of published guidance, standards, and expectations to help medical device and combination product developers navigate the expectations of Health Authorities.
A recent article published in the journal Expert Review of Medical Devices, however, sheds some rare and useful light onto this topic. Written based on research undertaken with the EU MDR's Clinical Evaluation Consultation Procedure (CECP) Expert Panels - specialist groups tasked with reviewing the Clinical Evaluation assessments prepared by Notified Bodies as part of the approvals process for certain high-risk devices - it describes findings from 34 CECP opinions from 281 submissions, providing insights into the common Clinical Evidence failures.
While the procedure is applied only to the highest risk product categories, the opinions and expectations outlined by the CECP panel help guide and shape the opinions of Notified bodies in general when reviewing Clinical Evaluation Reports, and so act as a broad guide to the kind topics likely to be raised by Notified Bodies reviewing lower risk devices. The article provides a useful guide as to what pitfalls to avoid in a Clinical Evaluation package to avoid unnecessary delays, cost, or rejection during the regulatory approval process.
The findings
The key challenges highlighted by the report are shown in the figure here, categorised by the type and number of comments made by the CECP expert panels across the 34 opinions
Summary of the comments from the expert panels' assessment of the Notified Bodies' review of Manufacturers' clinical evidence
The key challenges raised by the CECP expert panels fit into four broad categories:
The sufficiency of clinical evidence (quantity and quality of data) (40 comments)
The consistency of clinical evidence with the Post Market Clinical Follow-up Plan (25 comments)
The adequacy of the manufacturer's benefit-risk determination (10 comments)
The consistency of the clinical evidence with the stated intended purpose of the device (8 comments)
Let's dig a little deeper into each topic, and what the expert panels determined
1. Sufficiency of Clinical Evidence
The issues raised here related mostly to the quantity and quality of data provided. A common finding was that either the sample size for clinical studies or the follow-up time of the study was insufficient to make a proper determination on the safety and efficacy of the system. In addition, many comments related to the overall quality of the clinical evidence supplied, and limitations due to inadequate study designs and methodologies. This highlights the need for additional focus on the planning of clinical investigations, in particular that they are of long enough duration to cover the full lifetime of the device, and have sufficient statistical power to demonstrate the key safety and efficacy characteristics.
Incomplete literature surveys and inconsistencies within the submission we also discussed as challenges - again highlighting the needs for effective planning and completion of both the Clinical Evaluation Plan, and the Clinical Evaluation Report itself
2. Consistency of Clinical Evidence with PMCF Plan
Post Market Clinical Follow-up is a topic receiving greater regulatory scrutiny worldwide, and as included as a necessity within the EU with the introduction of MDR. Regulators don't only want to see the results of safety and efficacy studies in the controlled environment of a clinical study, but an effective plan to analyse how the system performs in the real world on-market.
Challenges with the PMCF plan were the second most commented topic by the CECP Expert Panels, and mirrored limitations found in the Clinical Evaluation Reports themselves - a lack of detail establishing the quality and quantity of PMCF plan actions is sufficient, including concerns around fit to the product lifetime; a lack of match with the claimed intended purpose/indication; and insufficient consideration of the target EU population.
Much like with the Clinical Evaluation itself, this highlights a need for additional focus on how the PMCF is designed and documented within the medical device or combination product manufacturer - to ensure that the specific testing proposed matches the real intended use of the system and adequately outlines the overall safety and efficacy of the product.
3. Adequacy of Benefit-Risk Determination
At the core of any Clinical Evaluation of a medical device or combination product is an assessment of the clinical benefit the intervention provides relative to the risk to the patient of said intervention. For a device to be approved, the benefit risk at a minimum needs to be positive - the system delivers more benefit than risks - and more generally needs to be shown to outperform existing/comparable interventions already on the market.
The most common challenge highlighted here by the CECP Expert Panels is not ascribing sufficient risk to the overall use of the device, and any adverse events that may have been experienced by patients during the course of clinical testing. The produces a benefit-risk outcome that is skewed too far towards benefit, based on the available evidence, and so requires either a re-assessment or additional clinical testing to demonstrate. Similarly an incomplete review of the state of the art was found by the expert panels to give too high a relative benefit-risk rating when compared to existing interventions.
Again, the lesson here for us as medical device developers is to ensure we assess risk clearly and objectively. Whereas a lower risk value can paint our device in a positive light, this is only that case if it adequately reflects the outcome of the clinical data gathered.
4. Consistency of Clinical Evidence with Intended Purpose
The anchor of any medical device or combination product is its intended purpose - the description of the medical function the system is intended to fulfill for patients. Clearly then, the clinical evidence generated and provided to Notified Bodies must be focused tightly on demonstrating the safety and efficacy of the device when used for this intended purpose when applied to the target patient population.
The expert panels found that for some of the cases, the clinical evidence provided did not support all of the intended claims of the device, and so overall approval could not be granted. The solution here is multi-faceted, in making sure that the all intended use claims are justified via the clinical evidence gathered and submitted as part of the regulatory submission.
In some cases this may mean being more honest with ourselves around what our medical system is capable of, and so trimming down intended use claims that are beneficial, but can't be substantiated with the clinical evidence we are ale to gather.
Addressing the issue from the other side, the planning and results of our clinical testing, as captured in our Clinical Evaluation Plan, and Clinical Evaluation Report, must be set up to specifically demonstrate the safety and efficacy of our system for all intended use claims.
Summary
First the good news, that there is nothing truly new here in how to successfully pull together a Clinical Evaluation Report. Instead the emphasis is on implementing the key elements better; ensuring we have the right endpoints that match a well-targeted intended use, sufficient statistical power in our studies, an objective assessment of risk (and risk benefit), and assessment over the lifetime of the intervention as priorities. A relatively newer element is the focus on effective PMCF plans, but these are also increasingly well established as MDR becomes embedded.
The good news here is, however, also bad news. Effective planning of Clinical Evaluation based on established approaches has been the target of industry all along. This means we are likely still missing something in the detail, either not having the deeper Clinical Expertise needed to make a judgement about, for instance, relevant endpoints, and sufficient statistical power. Alternatively the complexity of determining this is an inherently difficult problem that even expertise fails to get right every time.
For most devices the first case is the most likely however, putting the onus on all developers to make sure they have sufficient Clinical Expertise available in planning and executing their Clinical Evaluation work. In addition as a industry we could follow the recommendations of the paper's authors and agitate for clearer guidances and standards for what constitutes sufficient clinical evidence, making the overall process much more transparent and understandable.
Are you seeking help or advice in developing your Clinical Evaluation Plan or Clinical Evaluation Report? Please do contact us directly here for a no obligation 1-on-1 chat