.svg){kind=icon}
Under the Medical Device Regulation (2017/745) (MDR), demonstrating clinical benefit and quantifying benefit-risk ratios are critical to compliance. A medical device must not be placed on the market if the benefit of the product does not outweigh the risk in a clearly quantified and documented benefit-risk analysis. Qualitative arguments are inherently subjective to some degree – an issue that can be addressed by a quantitative approach.
However, despite the importance of benefit-risk ratios in showing conformity with the General Safety and Performance Requirements and placing a product on the market, there is currently little relevant guidance on how to effectively quantify benefit-risk and provide rigorous justification for the conclusions reached. The MDR does not define what an acceptable benefit-risk determination is or discuss how to justify the occurrence of residual risks. As a result, manufacturers are producing purely qualitative benefit-risk analyses, and these often do not meet the required quality level.To enable a quantitative analysis, it is important to begin by defining the relevant and appropriate benefits and risks for the device when used as intended. Improvements in patient health and possible harms need to be specific, and the outcomes must be measurable. Measurable indicators help to quantify the impact of a device, as the indicators can be used to compare a patient’s health before and after treatment, or to compare two different devices.
- The MDR defines benefit-risk determination as the analysis of all relevant assessments of benefit and risk when the device is used for its intended purpose (See Article 2 (24) for the complete definition). The regulation refers to the benefit-risk ratio as benefit-risk determination and benefit-risk analysis interchangeably.
- Benefit is defined in Article 2 (53) as the positive impact of a device on the health of an individual, expressed in terms of meaningful, measurable, patient-relevant clinical outcomes, including outcomes related to diagnosis, or a positive impact on patient management or public health.
Watch the Webinar: An Intuitive Approach to Quantifying the Benefit-Risk Ratio
Clinical Benefits
When identifying clinical benefits, the key principle is to approach them from the perspective of the patient or user, or in terms of patient management. Patient-related clinical benefits tend to be discussed mostly in measurable terms of improvements in quality of life, symptom relief, pain relief, reduced rates of re-interventions, improved patient management outcomes such as enhanced diagnosis (for imaging devices) and technical success in the facilitation of index procedures (for surgical accessory-type devices). There may be multiple benefits if the device has more than one positive impact. In sum, the benefit outcomes should tell the story of clinical improvement that the device under evaluation provides to the patient.
Clinical benefits should not be considered as synonymous with performance, since they may not necessarily be the same. Performance and benefit can be differentiated as follows:
| Performance | The ability of a device to achieve its intended purpose |
| Clinical Performance | The ability of a device to achieve its intended purpose, thereby leading to a clinical benefit |
| Clinical Benefit | The positive impact of a device on the health of an individual, expressed in terms of a meaningful, measurable, patient-relevant clinical outcome(s) |
Risks
ISO 14971 defines risks as the frequency of occurrence of a harm combined with the severity of the harm, the harm being injury or damage to the health of the patient.
Risks identified for the device in the clinical evaluation should align with the risk management file and other available risk documentation. The analysis should include the most common harms, including those identified in the instructions for use and those reported in clinical studies using the device. Ideally, only risks that can be reasonably attributed to the device under evaluation would be included in the analysis. However, it is prudent to include any adverse events reported that could reasonably be associated with the use of the device.
An illustrative example
We applied the principles above to an anonymized wound dressing, AdBan. Using the product’s instructions for use and risk management documentation, we differentiated between: performance outcomes and clinical benefits; and risks attributable to the device versus other adverse advents that could be associated with the product.
| Intended Use | AdBan Wound Dressing provides a moist environment for the management of partial and full thickness wounds. |
| Indications for Use | 1st and 2nd degree burns Traumatic wounds Surgical wounds Pressure ulcers Leg ulcers Diabetic foot ulcers |
| Performance Outcomes | Duration of adhesion Moisture level of wound environment Reduction of wound size |
| Benefits | Wound healing |
| Risks Attributed to the Device | Allergic reaction Periwound maceration Skin damage on removal |
| Other Risks | Bleeding Infection |
Next Steps
The next step is to determine overall benefit and risk for the device. There are two key factors in this assessment: magnitude and frequency. Essentially, the following questions need to be answered:
MAGNITUDE
- How great is the benefit?
- How severe is the risk?
FREQUENCY
- How many people experience this benefit?
- How many people experience this risk?
The rationale for this approach is clearly explained by the guidance document MEDDEV 2.7/1, Rev. 4, A7.2: “A large benefit, even if experienced by a small population, may be significant enough to outweigh risks, whereas a small benefit may not, unless experienced by a large population of subjects.”
Magnitude and Frequency Values
Frequency of occurrence may be easy to measure if the Specific and Measurable Outcome (SMO) already expresses the number of patients that experienced the benefit. However, in some cases, it may be necessary to set certain thresholds to then see the number of patients that achieved a specific level of benefit or risk. For example, wound healing could be measured by the reduction in wound size. If a reduction of at least 5 mm2 in the wound surface area is clinically relevant, then this can be the threshold for frequency of occurrence.
Determining magnitude values is a more subjective task. Some outcomes may seem to be well suited to a direct measure of magnitude. However, on closer glance, these measures may not necessarily provide clinically relevant information. For example, the decrease in wound surface area (measured in mm2) indicates the magnitude of wound healing and the amount of blood loss (measured in mL) indicates the severity of bleeding. But a 5 mm2 reduction in wound surface area may be much more beneficial than a 5 mL loss of blood is severe.
RQM+ has developed a novel method that reduces subjectivity and bias when reviewing magnitude and frequency. This method is used to: assign magnitude and severity values for outcomes which lack these implicit measures; and to establish a normalized scale so that benefit and risk can be directly compared. This method is outlined in more detail in our dedicated white paper: Benefit-Risk Determination: A Quantitative Approach.
Calculating Benefit-Risk Ratio
Next, by combining the frequency and magnitude measures, each benefit and risk can now be quantified as a single value. Once the benefit and risk values are determined, it is possible to produce a simple benefit-risk ratio.
The calculation looks like this:
| Benefit Value | = | Frequency x Magnitude |
| Risk Value | = | Frequency x Severity |
| Benefit-Risk ratio | = | Benefit Value __________________ Risk Value |
This method provides a value for each benefit and each risk; it does not combine all benefits nor all risks to establish one singular benefit value and one singular risk value. This is because a singular value would hide the detail needed for an in-depth analysis. For instance, there is a danger that high-severity risks may be obscured by many low-severity risks, if all risks are simply presented as one value.
Determining Acceptability
After this process of evaluation, how can manufacturers decide whether their benefit-risk ratios are acceptable? Acceptance criteria should be established before calculating benefit and risk values, so that acceptability can be argued in a clear, rigorous, and unbiased manner. In basic terms, any benefit-risk ratio greater than 1 is favorable (i.e., the benefit value is greater than the risk value).
However, the MDR also requires a comparison with devices or therapies that are generally accepted as state of the art. Benefit-risk ratios can therefore be calculated for the state-of-the-art, using the same methodology. This allows for a direct comparison; ideally, the benefit-risk ratio for the device under evaluation will be greater than that of state-of-the-art devices.
Acceptance criteria for each device may be unique, to address the specific characteristics of the device and target patient population. For example, if a device has multiple benefits and risks, there will be a collection of benefit-risk ratios to analyze, which makes it more challenging to define criteria. In these instances, a different approach may be needed. One option is to specify that the benefit-risk ratio for at least half of the benefit-risk pairs should be greater for the device under evaluation than for the state of the art.
More Resources From RQM+
The steps outlined above present one suggested method of calculating a benefit-risk ratio. RQM+ has successfully implemented this approach to satisfy requests for quantitative benefit-risk analysis from BSI. Our white paper on the topic includes an anonymized real-life example of how we put this approach into practice. You can also check out our related webinar, An Intuitive Approach to Quantifying the Benefit-Risk Ratio.
This is not to say that other methods are not possible. Ultimately, there must be sound justification for whichever method is selected. Whether you’d like support on calculating your benefit-risk ratios, or an unbiased review of your justification, book a consultation with the team here – we’re here to help!
.svg "white RQM logo")
