Yihui Xie

Obligation for Statistics to Draw Conclusions?

Yihui Xie / 2008-01-09

Of course it is NOT the obligation for statistics (or statisticians) to draw conclusions.

Statistics helps us to understand facts (approximately, in most cases), and the result might be “insignificant” or “unstable”, etc. I believe most people hate such results, as they cannot draw conclusions as usual.

A stranger sent me an email asking about an unstable result from his/her K-Means cluster analysis, and this guy wanted to know a “good” method to choose the initial centers so that the cluster membership would be stable, besides, he/she also wondered which result was “correct”, as there were many results according to different initial centers.

My answer to such a kind of questions is just the title for this blog. No matter how eagerly you want a “beautiful” result, please make sure you understand statistics first. “Insignificant” or “unstable” results are ALSO results, although they look different from what you read in most papers or textbooks. “Unstable” cluster membership just indicated that individuals in your sample were actually NOT clustered from the viewpoint of K-Means cluster algorithm. Who can deny this is NOT also a conclusion?