Assessing Prosthesis Performance

An important Registry focus has been the continued development of a standardised algorithm to identify any prosthesis not performing to the level of others in its class. This work is not readily apparent in the Report but is critical to its function. A pragmatic two-stage approach has been developed.

As currently implemented, the first stage is an automated system that selects for further attention any component where:

1. the revision rate (per 100 component years) exceeds twice that for the group, and
2. the Poisson probability of observing that number of revisions, given the rate of the group, is less than 0.05, and


either


3. there are at least 10 primary procedures for that component, or
4. the proportion revised is at least 75% and there have been at least 2 revisions.

Additionally, if a component represents more than 25% of the group, its revision rate is excluded from estimation of the group's overall rate. The purpose of this stage is to bring to early attention any prosthesis where there is a performance discrepancy.

In the second stage, the Director of the Registry, the Chairman of the AOA Registry Committee and the Coordinator of the Registry, in conjunction with staff of the Data Management and Analysis Centre, review the findings and decide if mention of a component in the Report is warranted.

Many factors are considered when making this decision. They include amongst others the relevance of the statistical significance of the observed higher revision rate and the presence or absence of any confounding factors. It is known that many different factors may affect the outcome and careful consideration must be given before any particular prosthesis is highlighted. To date a small number of the prostheses identified in the first stage of the algorithm have subsequently been highlighted in the Registry Report. The major reason for not including the majority of identified prostheses is inadequate numbers or the inability to exclude confounding factors. This algorithm will be subject to change as its performance is reviewed and further data are collected.


Survival Analysis

When the Registry describes the time to revision of a prosthesis using the Kaplan-Meier estimates of survivorship (See Annual Report) we show the curve only while the proportion of prostheses that is at risk for revision is at least 10% of the initial number at risk for that type. This avoids uninformative, imprecise estimates at the right tails of the distribution where the number of primary prostheses at risk is low. However, analytical comparisons of prostheses' survival using log-rank tests and proportional hazards models are based on all available data. (ref Pocock SJ, Clayton TC, Altman DG. Survival plots of time to event outcomes in clinical trials: good practice and pitfalls. Lancet 2002; 359: 1686-89).

Confidence intervals for the Kaplan-Meier estimates are point-wise Greenwood estimates and their appearance should not be used to infer whether overall differences in survival between prosthesis types are significant. Rather, the log-rank tests and hazard ratios reported with each curve should be used for this purpose.