DRMacIver's Notebook
Stories about Data
Stories about Data
Consider the difference between the following:
The result was non-significant but when we split it up into N groups it became significant for subgroup Y. Therefore there is a significant effect for group Y.
Versus:
The result was non-significant but when we split it up into N groups it became significant for subgroup Y. Obviously this is invalid statistically, but that might be an interesting followup experiment to perform.
It’s OK to tell whatever stories you want about your data, as long as you make it clear which ones are and aren’t valid inferences.