Implementing Equivalence Testing for the Evaluation of Parallelism: Insights from Dr. Todd Coffey

In today's blog post, Biological Assays presenter Dr. Todd Coffey, CMC Statistician, Seattle Genetics, share with us a  little about his work in evaluate parallelism for bioassays.  Here's what he had to share.

One of the criticisms of the equivalence approach for assessing similarity is that development teams will inadvertently utilize compromised samples and implement a too-wide “zone of indifference”. How do you protect against this problem?
Compiling a set of historical data that includes the natural variability of parallel curves is not a trivial exercise. I offer three suggestions to protect inadvertently against including compromised samples: 1) Ensure the dataset is large enough to include all sources of assay variation. With an adequately large dataset, patterns can often be identified that may elucidate which samples are compromised and why they should not be included with the other parallel samples. 2) Carefully assess the data for unusual trends and patterns, both visually and with statistical analysis. 3) Compare the parallelism metric for degraded samples that are expected to be non-parallel to parallel samples that are suspected of being compromised.

The USP states that one can use the absolute difference of slopes or a ratio of slopes when using the equivalency approach for similarity. Do you have a preference and why?
Using ratios has the advantage of being generalizable across assays. However, calculating confidence intervals on the ratios is not always straight-forward because the ratio of two normally distributed variables is not normally distributed. Thus special care has to be used to correctly calculate confidence intervals on ratios. While not as generalizable across assays, calculating differences between standard and test is more straight-forward statistically and is also interpretable. For these reasons, I prefer to set equivalence limits using differences

How many reference vs. reference runs do you recommend using to establish a “zone of indifference”?
There are at least three issues to consider when discussing sample size. First, to be representative, the number of runs needs to include all sources of variation. Second, the sources of variation have a great impact on the relative value of measurements and the sample size is dependent on getting the right data. For example, if most of the assay variation comes from factors that vary between runs, then many measurements in the same run are of much less value than measurements from different assays. In this case, the sample size is dependent more on the number of times the assay is run after varying the factors that cause the variation. Finally, to provide accurate estimates of tolerance intervals, the sample size of independent measurements generally needs to be several dozen. When that sample size is not attainable, I recommend setting initial limits, monitoring the assay, and then modifying limits as new data emerge.

Many companies start with a difference approach to similarity during early product development when they have a small number of lots and the assay is still being developed. At what stage of development is it reasonable to implement an equivalency approach for similarity?
I recommend using the equivalence approach when there is an adequate set of representative historical data that contains all sources of variation. Sometimes this dataset is available during qualification or before the IND is submitted. If it is not available then, the next potential milestone may be when process characterization activities begin.

Dr. Coffey will be presenting Tips and Tricks for Implementing Equivalence Testing for the Evaluation of Parallelism this May 14-16, 2013 in Seattle, Washington at the Development, Validation and Maintenance of Biological Assays event.  For more information on his session and the rest of the program, download the agenda.  If you'd like to join him, as a reader of this blog when you register to join us and mention code IBA13JP and save 20% off the standard rate.  Have any questions?  Feel free to email Jennifer Pereira.

Share this article with your social network, just click below to share now!

Share |

The Analyzation of SPC Data

Stanley Deming, Ph.D., President, Statistical Designs recently sat down with us to go over some of the topics he'll be presenting on during the Development, Validation and Maintenance of Biological Assays event taking place this coming May.

Today, Dr. Deming answers the question:
Are there differences in how SPC data should be analyzed if one is interested in spotting emerging trends rather than determining the acceptability of a particular assay?

This is, in part, a trick question: SPC data should never be used to determine the acceptability of a particular assay. Concepts like “system suitability criteria” and “assay acceptance criteria” are essentially “specifications” (whether we call them that or not), and it is a commonplace that “specifications should be based on fitness for use”, not on the behavior of the assay. 

Statistical process control limits can be thought of as a visualization of the voice of the process (assay) talking to us and telling us how it is behaving. Specification limits can be thought of as a visualization of our voice talking to the process and trying to tell it how we want it to behave. Unfortunately, the process could care less about what we want – it is not going to listen to us, it is just going to do what it does. 

Setting specifications based on statistical process control limits is misguided thinking – control limits and specification limits are totally separate concepts. For example, a process can be out of statistical control and still be within specifications (i.e. fit for use). Statistical process control limits should never be used to set specification limits – specification limits should be based on fitness for use. 

Getting back to the first part of the question: The “rule of eight” (or, as I prefer, the “rule of ten”) states that if eight (or ten) or more consecutive data points fall on the same side of the center line on either the x-bar or r chart, then the process is out of control in the sense that it is drifting off in one direction. This indicates an ‘emerging trend” and should be brought to the attention of the assayist. 

As a final point, sometimes “emerging trends” are not necessarily bad. As an example, if the r-chart has eight (or ten) or more consecutive data points below the centerline, this is an indication of improved precision. This suggests that there is an opportunity to discover why the precision has improved, and to then implement that discovery so the precision can remain improved in the future.

Dr. Deming will be leading the workshop Process Capability and Variance Components Analysis on May 14, 2013.  For more information on this session and the rest of the program, download the Bioassays agenda. If you'd like to join us May 14-16, 2013 in Seattle, as a reader of this blog, when you register to join us and mention code IBA13JP, you'll save 20% off the standard rate! Have any questions? Feel free to email Jennifer Pereira.

Share this article with your social network, just click below to share now!

Share |