Steps in Conducting a Successful DoE Experiment
Conducting a DoE experiment involves using many of the tools of six sigma quality that were outlined in previous chapters. It is always advantageous to form a team to perform the tasks of designing the experiments and interpreting the results. Teams have shared experiences in the design, and can achieve broad consensus on different approaches to the DoE and the problem being analyzed.
The success of a DoE project is dependent on selecting the proper team members, identifying the correct factors and levels, focusing on optimizing and measuring the quality characteristics, and analyzing the results. Steps in performing a successful robust design of experiments are as follows:
• Problem definition. The first task in performing a DoE project is to outline the goals of the project and to define the quality characteristics of process or the design to be optimized. Although only one characteristic can be optimized at a time, many characteristics can be measured from the same experiment matrix while performing the experiments and analyzed separately. The final-level selection can be a mix of the recommended factor and level settings，depending on the compromise of the different objectives of each quality characteristic.
• Design space. Creating the boundary of the product or design to be optimized is important. The experiment should not be constrained to a small part of the design and hence not provide the opportunity to study the interactions between the different parts of the total design. On the other hand, the experiment should not be all-encompassing in an attempt to optimize a wide span of product design, steps or processes. Ideally, the total design should be analyzed, an a compromise made in developing a plan for a succession of DOEs each providing additional information about the design to be optimized.
Team creation and dynamics，A project team should be selected to conduct the experiments and perform the analysis. The team should be composed of those knowledgeable in the product and process, and should solicit inputs from all parties involved in the design to be optimized. It is not necessary to have an in-depth technical understanding of the science or technology of the problem, but the team members should have experience in similar or previous designs.
Knowledge in statistical methods, and in particular DoE techniques, should be available within the team，either through a statistician or someone having received training or experience in DoE.
• Factor and level selections. DoEs can be performed using two approaches. One method is to select a large number of factors and use a screening experiment, usually a saturated design (to be explained later), to narrow down the factor selections. Then a follow-on experiment, preferably a full factorial experiment, is used to complete the selection of the optimal factors and their levels. The second method is to have the team members consider this DoE project as a single opportunity to try out as many possible factors, levels, and combinations of both, because of the lack of time or resources available. In this case, partial factorial experiments are used, with some assumptions as to the relationships of factors, in order to maximize the benefits, resources, and time spent on a single experiment.
• Brainstorming techniques should be used to select the number of factors, and the different levels for each factor. The selection process should outline factors that are as independent as possible from other factors, and hence are additive in controlling the quality characteristic(s) to be optimized. This is important in reducing the interactions of factors, which are difficult to quantify statistically.
An example of selecting independent factors and reducing their interactions is the case of an infrared conveyorized oven for the reflowing surface mount technology printed circuit boards (SM1 The reflow process is characterized by three factors: the ramp-up of temperature to the solder melting stage, the maximum temperature level reached during reflow, and the time during which the temperature remains above the solder liquid state, usually called time above liquidus(TAL). There are several heater zones in the reflow oven, and the oven temperature can be controlled by setting the zones on the top and bottom of the reflow oven to predetermined levels, as well as varying the conveyor speed. Choosing temperature zones and the conveyor speed as the factors for a reflow experiment would result in strong interaction between the factors. The proper choice of factors would be the ramp-up temperature rate, the TAL, and the maximum reflow temperature. The factor levels selected should be achieved by actually experimenting with the temperature zones and conveyer speed to reach the desired levels in the experiment.
Level selection. Proper selection of the levels for each factor used in the experiments is important in achieving the proper design Space- Levels that are either too close together or too far apart in value should not be selected, because they do not represent a continuum of the impact of the factor on the measured characteristic. Level selection should follow these guidelines:
1. Three level designs could be chosen if the project team is confident that the current design is performing adequately but needs to be improved. The current level should be in the center of a 20% span represented by the other two levels. In this manner, the DoE can help in finding a more optimized operating set of factors levels in the design space.
2. Two levels could be selected if there is little confidence in the adequacy of the current design, based on the collective judgment of the team. By choosing two levels, more factors can be tested within a small number of experiments, as will be demonstrated later. In addition, the direction of better design performance can be ascertained for future DoEs.
3. Multiple level factors should be chosen for survey experiments. In these DoEs, a team can select many new technologies or materials within one factor to identify which one can perform best in the design. The number of multiple levels should be close to squares of two or three levels, such as four, eight, or nine levels. They are easier to perform since they fit easily into the set of predetermined experiment arrays.
4. The selected levels should be well within the operating range of a working characteristic within the design space. In the soldering reflow experiment mentioned above, the combination of temperature factors and levels should not result in having components soldered beyond their maximum temperature and time exposure specifications.
• Experiment arrays. Most DoE experiments use a set of standard orthogonal arrays available to conduct the experiment, with two 〇r three levels. There are only certain combinations of factors and their levels available in order to perform the experiment. Compromise might be necessary to achieve economy in DoEs by selecting a given number of factors and levels that can fit within one of the orthogonal arrays. There are only a small number of these arrays of two and three levels, and their size increases geometrically with the number of factors selected.
• Conducting the experiments is based on the selected orthogonal arrays. The arrays are arranged in terms of the number of experiments, factors, and levels. The experiments should be conducted in a random order from the array matrix. The measurements of the characteristic to be optimized could be repeated using various scenarios, depending, on the variability considerations of the design (see the later section on variability reduction).
• Data analysis. Once the experiments are performed, the data can be analyzed graphically to determine the optimal settings of levels of significant factors. In addition, statistical analysis can be performed in order to determine the significance of each factor's effect on the quality characteristic, through the use of analysis of variance (ANOVA). Important factors can be set to the proper level, and least significant factors can be ignored, or set to the most economic conditions.
• Graphical analysis of the data is sufficient to determine the best factor setting to adjust the design average to target and reduce design variability. The statistical analysis provides more details on the probability of the effect of each factor on the characteristic measurement. In addition, statistical analysis can quantify the usefulness of the DoE project: low significance of the total experiment usually results from the lack of significant factors. In this case, the experiment is not providing useful guidance to the design team and it should be repeated with additional or different factors and levels.
• Prediction and confirming experiments. Once the graphical and statistical analysis is completed, the characteristic value can be predicted based on the choice of factor levels. These choices could be a compromise between setting the design characteristic average to the target value versus reducing variability. A recommended factor level might cause variability to be reduced, yet at the same time the process average will be shifted from target. Another case is when multiple characteristics are to be optimized using one experiment with many separate output measurements and data analysis. For example, a robust design experiment could be performed to de sign a new plastic material to be injected molded. The material and process design can have several desired characteristics including modulus of elasticity, density, amount of flash after injection gel time, flow rate, and free rise density. A DoE experiment could be designed using an orthogonal array that determines what ratios and composition of raw materials are to be used, as well as the injection molding machine parameters. Measurement of all the desired characteristics will be performed, then the data analyzed to determine the best set of raw material ratios for each characteristic. A compromise of all recommended factor levels will have to be made in order to achieve the best overall plastic product.
• Confirmation experiment. Once all the choices and predictions of the DoE experiment have been agreed upon, a confirmation experiment run should be made before final adoption of the design decision, to verify the analysis outcome. This confirmation will test the entire robust design process before full implementation takes place. In manufacturing, the newly adjusted process should continue to be monitored through statistical quality control methods for a six month minimum time period, before any attempts are made to further increase the robustness of the process by launching another DoE.
