The numerical compearing of classical and permutation methods of statistical hypothesis testing

Abstract

The article is devoted to the classical problem of a statistical hypothesis test for the equality of two distributions. For normal distributions in many ways t-test is the best one. But in practice, compared distributions often are not normal ones and, generally speaking, are not known. The non-parametric Kolmogorov—Smirnov test is usually used to solve this problem in case when the investigator knows nothing about the compared distributions. The article deals with methods based on permutations. In recent years such methods have attracted attention by its simplicity, versatility and relatively high efficiency. A comparative study of power of a few permutation tests and classical methods (such as Kolmogorov— Smirnov test, t-test and Mann—Whitney test) for a wide class of distribution functions was done by using stochastic simulation’s methods. Normal and Cauchy distributions and its mixtures are considered as well as exponential, Weibull, Fisher and Student’s distributions. It was found that for many typical distributions the permutation test, based on the sum of the absolute values of the differencesis, is the most powerful one. The advantage of this test over the others is considerably greater for the case, when symmetrical distributions with the same centers are compared. Thus, the permutation test can be recommended for using in cases where the compared distributions are different from normal ones. Refs 9. Tables 5.