OptiGein Design Docs
Home › Diagnostics

Diagnostics

The ANOVA tells you the model fits. Diagnostics tell you whether you are allowed to believe it. This tab checks the assumptions behind every p-value, so you catch a bad run or a wrong model before it reaches a report.

Diagnostics is the fifth tab on an analysis page, after Configure, Fit Summary, Model and ANOVA. It exists because least squares makes promises: the residuals are random, they have the same spread everywhere, and they follow a normal distribution. When those promises hold, the ANOVA is trustworthy. When they do not, a small p-value can be an illusion. Every plot here is one way of asking whether the promises hold for your data.

The tab is a grid of four resizable plot panes plus a Report table. Each pane has its own sub-tabs, so a lot of graphs live in a small space. Every point is coloured by its response value, from blue at the low end to red at the high end, which helps you see whether trouble is concentrated in the big or the small results. Hover any point to read its exact numbers. All of this is recomputed from your data the moment you press Start Analysis on the Configure tab.

The Diagnostics tab showing the normal plot, residual plots, Box-Cox and influence measures
The Diagnostics tab for Disintegration Time in the worked HPMC E15 and Glycerol Ester example. Four plot panes, each with sub-tabs, and a Report tab.
One word first: studentized. Most of these plots use the externally studentized residual, not the raw residual. That is the residual for a run expressed in standard deviations, using an error estimate that leaves that same run out of the calculation, so a genuine outlier cannot quietly shrink the yardstick it is being measured against.

Normal plot of residuals

This is the first assumption to check. The plot puts the externally studentized residuals on the horizontal axis and their normal probability on the vertical axis, with the vertical scale stretched so that a normal distribution comes out as a straight line. A reference line is drawn for you to compare against.

Residuals versus predicted, run and factor

The second pane holds three related plots, each with the externally studentized residual on the vertical axis and a zero line through the middle. Red control lines mark the outlier limit: any point beyond them is worth a look. The limit is a Bonferroni corrected t value, computed from your design rather than fixed in advance. It takes account of the number of runs and the residual degrees of freedom, so the cutoff fits the size of your experiment instead of being one blanket number.

Box-Cox plot for a transform

When the plots above suggest a transform, the Box-Cox plot tells you which one. It refits the model at many powers of the response and plots the natural log of the residual sum of squares against the power, called lambda. The lowest point on the curve is the transform the data likes best.

Three markers make it readable. A blue line sits at your current lambda, which is the transform set on the Configure tab. Out of the box the response is untransformed, so the current lambda is 1. A green line sits at the best lambda, the value at the bottom of the curve. Two red lines mark a 95 percent confidence interval around the best value. The text panel on the left states the current lambda and a plain recommendation, such as Natural Log or Square Root or None.

How to read it: if the blue current line falls inside the red interval, no transform is needed and the recommendation reads None. If the blue line sits outside the interval, the response would benefit from the transform named in the panel. Box-Cox needs every response value to be positive, so if a response contains a zero or a negative number the plot explains that instead of drawing a curve.

Read the recommendation as advice, not an order. A transform can straighten the normal plot and flatten a megaphone in one move, but it also changes how the model reads. Only reach for one when a diagnostic actually points to it, not out of habit.

Predicted versus actual

This plot, shared with the Box-Cox pane, is the simplest honesty check in the tab. It puts your measured values on the horizontal axis and the model's predictions on the vertical axis, with a diagonal reference line for a perfect fit. Points hugging the diagonal mean the model predicts what you actually saw. A point that sits well off the line is one the model struggles with, and its colour tells you at once whether the misses are among the high or the low results.

Predicted versus actual plot with points near the diagonal reference line
Predicted versus actual for the worked example. Tight clustering along the diagonal is a good sign.

Influence: who is steering the model

A model is fitted to all the runs at once, but not every run pulls with equal force. The fourth pane holds four influence measures, each plotted against run number with a threshold line that the software draws for you. A point past its line is not automatically wrong; it is simply a run with enough sway that you should know it is there.

MeasureWhat it asksLine the software draws
Cook's DistanceHow much every prediction would shift if this one run were removed.A flag at 1.0.
LeverageHow unusual this run's factor settings are, on their own, before the response is even looked at.Twice the average leverage, 2p divided by n, capped at 1, where p is the number of model terms and n the number of runs.
DFFITSHow far this run's own prediction moves when the run is left out, in standard errors.Plus and minus 2 times the square root of p divided by n.
DFBETASHow much one chosen coefficient changes when this run is left out. Pick the term from the box.Plus and minus 2 divided by the square root of n.

Leverage deserves a word. In the worked face-centred central composite design, the runs sitting on the edges of the factor ranges naturally carry more leverage than the five centre points, because their settings are further from the middle. High leverage on its own is a feature of a good design, not a fault. It only becomes a worry when a high leverage run also has a large residual, because then it is both influential and poorly fitted.

What to do when a point is flagged

A flag is a question, not a verdict. Work through it in this order.

  1. Check the data first. Open the run sheet and look at that run. A transposed digit, a wrong unit, a value typed into the wrong response: these account for most flagged points and are fixed in seconds.
  2. Ask what happened at the bench. If the number is typed correctly, think back to that run. A spill, a mislabelled sample or an instrument that misbehaved is a real reason to treat the run with suspicion.
  3. Repeat the run if you can. The cleanest resolution to a doubtful point is to run it again and see which value is real.
  4. Only then consider setting it aside, and say so plainly in your Notes. Removing a run changes every result on every tab.
Do not delete a run because it is inconvenient. A point that disagrees with your expectation is often the most informative one in the whole design. Remove data only when you have a specific, documented reason the run is invalid, never merely to make a p-value cross a threshold.

The Report table

The Report sub-tab lays out every per run diagnostic as one row per run, so you can scan the whole design at once and match a flagged plot point to its actual numbers. The columns are Run Order, Actual Value, Predicted Value, Residual, Leverage, Internally Studentized, Externally Studentized, Cook's Distance, DFFITS and Std Order. Two columns highlight problems in red for you: the externally studentized residual when it passes the outlier limit, and DFFITS when it passes its threshold. Those red cells are your shortlist of runs to investigate.

The diagnostics Report table with one row per run and flagged values shown in red
The Report table. Red values mark runs that crossed a threshold and are worth a second look.

When the plots are clean, the normal plot is straight, the residuals are an even band, no point is flagged and Box-Cox recommends None, you can trust the ANOVA and move on. The next tab draws the model you have just validated.