Experiments 6 - Wrap-up: the course in review, multiple objectives, and references for the future
Channel: Kevin Dunn
Date: 2015-08-19

Videos used in the Coursera course: Experimentation for Improvement. Join the course for FREE at https://www.coursera.org/learn/experimentation

These videos are also part of the free online book, "Process Improvement using Data", http://yint.org/pid

Full script for the video: http://yint.org/scripts/6
--------------------
We will give a high level overview, of the concepts we have covered in this course related to Design of Experiments for Improvement. We will also mention some topics we did not cover. 

The area of experimental design, is fairly broad, and the concepts we omitted, can quickly become very mathematical. But, if you choose to study these topics on your own afterwards, you will see that they build on the ideas we did cover in this course. 

You now have a solid foundation to base your self-learning on. All further, advanced experimental tools build on these concepts. 

We started the course by looking at some basic terminology: factors, outcomes, variables, low and high levels, and so forth. Very quickly we learned how to interpret - visually - the results from an experiment. And that was crucial; a visual interpretation is so important, and not having to run and rely on software. 

This is a theme we've seen throughout the course. We've resorted to visual tools the entire way. Cube plots, to visualize the results; Pareto plots to identify, or screen out factors, that appear important and those that are not.
And finally, in the response surface module we looked at contour plot, to visualize the surface we are moving on.

We also learned along the way, at several point how NOT to run an experiment. Changing one factor at a time, is something we have known for at least the last 8 decades as being inefficient, especially if we want to learn about and exploit interactions to reach that optimum. 

We learned how to set up our standard order table, to assist us. There are 2 to the k experiments in a full factorial. And once that full factorial was run, we saw how to manually create a simple prediction model. Remember that "high minus low", "high minus low" idea? No software was required. 

So by the end of the second module we saw that experiments often had more than one outcome. We might want to decrease pollution as much as possible, but also do so cheaply, and obey safety, or regulatory constraints. 

In systems where this is the case, we must either reformulate our objective to include multiple outcomes - maybe by using a weighted sum, for example - or by visualizing the overlapping, competing criteria on two contour plots. This visual approach is, again, very effective. We can see the trade offs in our system, and communicate with our colleagues effectively that don't understand the terminology of response surfaces and optimization. Furthermore, if things change in our system, we can quickly see how to compensate for them. 

Now in the third module of the course, we started using software, to speed up our hand calculations. We used a high quality, freely available tool to do that. The R software has many packages available to extend its functionality. But there are though other software tools, and some that are specifically designed for experimental analysis, feel free to download their trial versions and test them out for your own needs. 

We liked R, because of its traceability in the code. We can always go back, and reproduce our results. See where we've made mistakes and even share that code with our colleagues. 
You might be wondering about formal statistical tools that you might use to make your work more analytically: such as p-values, confidence intervals, analysis of variance, and so on. These are absolutely available, and have been there all along in the R output. As you've seen, we've been far more reliant on visual tools in this course, and less so on detailed statistical knowledge. 

In the fourth module of the course we started to look at fractional factorials. We use these when we have a large number of factors, and want to practically reduce the number of experiments to some lower value. We know that there's no free lunch, and that aliasing will occur. But we have this trade off table to help guide us in that choice. 

We learned about blocking for nuisance factors, and we also covered the idea of covariates in that fourth module. I had also mentioned the concept of definitive screening designs, which are emerging as a more effective design than fractional factorials. 

Perhaps this is a good time, to mention the book by Peter Goos and Bradley Jones. That book starts where this course ends. It's a great book, written in conversational style, that would help you peer into the minds of statisticians as they actually plan complex experiments. They cover topics, that many of you have asked about: response surface methods with categorical factors, screening designs, mixture designs, blocking and covariates, as well as the very practical requirements of a split plot design. ...