SPC Pillar: Design of Experiments (DoE)

This article concludes a series about statistical process control (SPC).

Check out the previous articles:

• What is Statistical Process Control (SPC)?
• SPC Pillar: Data Collection
• SPC Pillar: Using Control Charts
• SPC Pillar: Continual Improvement Process (CIP)

---

 

What is DoE?

Design of experiments (DoE) is a broad field, and this article will only touch on a few of the key features. DoE involves the planning and implementation of well-thought-out experiments and careful record-keeping to ensure that any conclusions reached through these experiments represent reality. Experiments are designed with statistics in mind so that the data collected can be analyzed with the relevant statistical tools.

In the average college laboratory class, students design very few of the performed experiments, instead provided with a set of steps or a recipe to follow. First and second-year students are provided these steps for safety reasons and for learning fundamental concepts about the topic. By the third and fourth year, students may dabble in developing their own experiments, but often have no methodology for designing these experiments. This leads to unnecessary experiments that don’t actually reveal much about the inner workings of the problem.

 

Figure 1. Experiments are the basis for predicting and testing the significance of statistical data we intend to use for making decisions. Image used courtesy of SimpliLearn

 

As engineers working in industry, there is no time to waste on experiments with no resulting value, or results based on poorly-collected data. Unlike the undergraduate laboratory, it is difficult to take a machine out of production to collect more data, if the first experiment batch didn’t collect the relevant data required.

Early Steps

Before designing an experiment, the engineer should first determine which quantities should be measured and which parameters can be controlled. This means spending some time on the factory floor to work with operators and take careful notes about their experience and observations. It may mean coming into work during different shifts to observe whether certain operations are performed consistently between shifts. It can be surprising how often multiple shifts incorrectly think they are performing the same steps in the same order.

After the procedures and controls are understood, the engineer can parse through and see if there are any “screening” experiments that may be missing. These are low-impact experiments that can help specify valid ranges for some of the controls. For example, suppose a valve is specified as a percentage open or closed. Because this valve may not behave linearly, it might be worth running the valve in several positions so that the system’s flow rate is understood fully.

Factorial Analysis

One tool for designing experiments is factorial analysis. It’s not the only tool, but it is often a good starting point for determining optimal process parameters. It is especially useful when there is a well-established process with an opportunity to optimize a particular property.

In factorial analysis, two or three controls are varied and their effects on a single output parameter are evaluated. If two controls are chosen, the experiment is two-dimensional (2D); if three are chosen, the experiment is three-dimensional (3D). All other controls are set constant during these experiments. The controls will vary from the largest and smallest values, as well as one “center” value.

As an example, consider a chemical vapor deposition (CVD) system which produces coatings. The thickness of the coating varies with the flow rate of a gas and the amount of radio frequency (RF) in the chamber.

 

Figure 2. This 2D factorial analysis shows a graph of the experiments to conduct. Each circle represents an experiment: the bottom left represents the lowest RF power and lowest gas flow rate, and the upper right represents the highest RF power and the highest gas flow rate.

 

After each of these experiments have been conducted, and the coating thickness measured, the engineer can look for trends. From there, they can set up new experiments with a new factorial analysis.

 

Figure 3. Suppose the two circled experiments provided the thickest coating. A new 2D factorial experiment can be created in the upper-right quadrant of this graph, if time and resources allow.

 

The real advantage of factorial analysis is that new information is acquired with each set of experiments. If time and resources only allow for five trials, the engineer can identify a region of optimized values. If the data is statistically insignificant, this can be noted as well since further experiments are required. However, if more time and resources become available, they can narrow the range of control values and create a new set of experiments.

 

Figure 4. New detailed experiments are represented by the small circles.

Contrast this with simply rastering through a list of process parameters, holding all control variable values constant, while stepping through adjusting one parameter at a time. It would be really easy to conduct five experiments and learn nothing!

Repetitions and Randomization

Five simple trials are rarely enough to make major process control decisions. Suppose instead, there was time for fifteen experiments. One could perhaps do three experiments at each of the five points on the factorial analysis graph. From there, the average at each point could be evaluated. This would be particularly valuable if the variability at each of the points was small.

The temptation, in this case, would be to conduct all three trials at a specific operating point before moving to the next. Don’t do this! Randomize all fifteen trials and document the trial run order. Using statistical tools, such as a Z-test or ANOVA, the run order can be checked to ensure that all experimental results are statistically significant.

In a different example, a beginning weightlifter wants to check how protein and water intake affect his lifting ability. Having never been to the gym before, he sets up a factorial analysis chart and begins his experiment. When he checks the run order after weeks of gym visits, a strange thing happens: the last experiments were his best, and it was statistically significant! We could analyze that obviously he got better at weightlifting, and this may have masked any changes in water and protein intake.

Final Thoughts

DoE is a method of making process control decisions based on data that was collected with a statistical plan in mind. It is a way to show that conclusions were derived from meaningful experiments, even in the fast-paced industrial world, where time and resources for conducting experiments are limited.

Statistical process control is one of the most important tasks for improving output, quality, and reliability of machines and processes. The steps to obtaining a better system are tested by time, and although each manufacturing process is unique, there is consistency in the system used to arrive at the final conclusions that will surely improve any business decision.