P3: DOE

 Welcome back to the blog!

This week, we went through Design of Experiment (DOE). This is a type of methodology where one "obtains knowledge of complex, multi-variable process with the fewest trials possible". This is something rather familiar to me as I had done this previously in Year 1 for my practical sessions.

DOE is comprised of a response/dependent variable, factors/independent variables, level, and treatment/run. I'll explain each one in detail:

Response/dependent variable - One variable that will be measured for a given experiment

Factors/Independent variables - variables that are changed to see its effect on the response variable. Usually there are only 3 chosen factors so as to restrict the number of runs to a minimum.

Level - Often referred to as high or low level. These are set values/behaviours for the chosen factors.

Treatment/Run - The experiment is run with a specific set level of factors.

Experiments themselves can be split between two mindsets: full factorial method or fractional factorial method. 

A full factorial method tests every kind of variable combination. In this case, 8 total runs are required for full factorial method for 3 given factors at either high or low.

A fractional factorial method tests only a subset of variable combinations. Usually for 3 given factors, 4 runs are required. This is often very useful for a quick observation of effects from each factor or when there is a big constraint such as limited allowed testing. For this method, it is imperative to have 4 runs that can create a non-bias in either 3 factors. This means that each factor must appear at most twice. This refers to statistical orthogonality.

With this knowledge in mind, we were tasked to investigate the effect of 3 factors that affect a catapult's flown distance. These 3 factors are: (including their notation and levels)

Factor A - Start Degree of catapult. High (+) = 35 degrees, Low (-) = 0 degrees

Factor B - Stop Degree of catapult. High (+) = 90 degrees, Low (-) = 60 degrees

Factor C - Arm length of catapult. High (+) = 31cm, Low (-) = 25cm. For this experiment, we used an alternative arm that was 3D printed.

For each run, we did 8 trials/rounds using the same parameters so that there is accuracy in our testing.

We then carried out the experiment using our set-up as shown below. We placed a tape measure on the near-end of the shown table so that we can get our readings. The catapult will shoot out the pellet, ideally landing on the sand at the bottom. The first dent would be measured. 

We had two problems. Firstly, our pellets would fly away so much that it would be difficult to retrieve them in the lab. We then tried to use the boxes' cover to stop it from bouncing outwards off the table. We also encountered a problem with distance being too high that our tape would no longer be able to measure it. We then moved the tape measure to approximately 80cm above the starting line instead so that we can get those data points. At the end we recorded and added each data point by 80cm.

We first did the full factorial method. We did 8 rounds of 8 runs for a total of 64 datapoints. These were then plugged into our trusty Excel to form this data table:

Using these averages shown, we determine the significance of each factor through each level. The average of each factor's level is calculated. For example, for factor A, one average is calculated only for runs containing high A (+A). Another average is calculated only for runs containing low A (-A). The averages were calculated as such:

With this, we plot the graph for each level of each factor. Using factor A again, it's like plotting a graph from the data point found in -A to the data point found in +A. Here it is:

The graph shows the significance of each factor through its gradient. Factor A has the highest influence, followed by factor B, while factor C has the lowest influence on catapult flown distance.

The gradient also shows the overall effect of each factor on the catapult flown distance. For a higher catapult flown distance, one must lower factor A, B and C.

We also did fractional factorial method. We used the same apparatus and set-up as before, however we only did 4 runs, for a total of 32 rounds. Each run set-up is shown in this table:

It is worth mentioning that this is statistically good as each factor appears at most twice within the 4 runs!

We do the same thing with these 4 runs and calculate the average of each factor's level:

With that, here's the plot for fractional factorial:

In terms of individual significance, it can be seen that factor B is the most influential, followed by factor A, and factor C being the least influential.

In terms of effect, for a higher catapult distance, one must lower factor A and B, while increasing factor C.

Wait... There are differences between full and fractional factorial methods! The effect and significance are both different.

That's the setback with fractional factorial method. We often would not be able to see the complete picture with just these 4 runs. However, these differences seem to be noteworthy, yet minimal if we only need to quickly see their effects.

 

Through this experiment, I found that full factorial method seems to be a sure-fire way to get the correct correlations, however fractional factorial method is a quicker and easier way. I personally prefer fractional factorial method, but I find that if I had the time and resources, full factorial method would be the best.

Thank you for reading!