Comparison of Some Methods of Testing Statistical Hypotheses: (Part I. Parallel Methods)


  • K.J. Kachiashvili Georgian Technical University, 77, st. Kostava, Tbilisi, 0175, Georgia, I. Vekua Institute of Applied Mathematics of the Tbilisi State University, 2, st. University, Tbilisi, 0179, Georgia



Hypotheses testing, -value, likelihood ratio, frequentist approaches, Bayesian approach, constrained Bayesian method, decision regions


The article focuses on the discussion of basic approaches to hypotheses testing, which are Fisher, Jeffreys, Neyman, Berger approaches and a new one proposed by the author of this paper and called the constrained Bayesian method (CBM). Wald and Berger sequential tests and the test based on CBM are presented also. The positive and negative aspects of these approaches are considered on the basis of computed examples. Namely, it is shown that CBM has all positive characteristics of the above-listed methods. It is a data-dependent measure like Fisher’s test for making a decision, uses a posteriori probabilities like the Jeffreys test and computes error probabilities Type I and Type II like the Neyman-Pearson’s approach does. Combination of these properties assigns new properties to the decision regions of the offered method. In CBM the observation space contains regions for making the decision and regions for no-making the decision. The regions for no-making the decision are separated into the regions of impossibility of making a decision and the regions of impossibility of making a unique decision. These properties bring the statistical hypotheses testing rule in CBM much closer to the everyday decision-making rule when, at shortage of necessary information, the acceptance of one of made suppositions is not compulsory. Computed practical examples clearly demonstrate high quality and reliability of CBM. In critical situations, when other tests give opposite decisions, it gives the most logical decision. Moreover, for any information on the basis of which the decision is made, the set of error probabilities is defined for which the decision with given reliability is possible.

Author Biography

K.J. Kachiashvili, Georgian Technical University, 77, st. Kostava, Tbilisi, 0175, Georgia, I. Vekua Institute of Applied Mathematics of the Tbilisi State University, 2, st. University, Tbilisi, 0179, Georgia

Georgia, I. Vekua Institute of Applied Mathematics of the Tbilisi State University


Bauer P. et al. Multiple Hypothesenprüfung, (Multiple Hypotheses Testing), Berlin: Springer-Verlag (In German and English) 1988.

Berger JO. Could Fisher, Jeffreys and Neyman have Agreed on Testing? Statist Sci 2003; 18: 1-32.

Berger JO, Boukai B, Wang Y. Unified Frequentist and Bayesian Testing of a Precise Hypothesis. Statist Sci 1997; 12(3): 133-160.

Berger JO, Boukai B, Wang Y. Simultaneous Bayesian-frequentist sequential testing of nested hypotheses. Biometrika 1999; 86: 79-92.

Berger JO, Brown LD, Wolpert RL. A Unified Conditional Frequentist and Bayesian Test for Fixed and Sequential Simple Hypothesis Testing. Ann Statist 1994; 22(4): 1787-1807.

Berger JO, Delampady M. Testing precise hypothesis (with discussion). Statist Sci 1987; 2: 317-352.

Berger JO, Sellke T. Testing a point null hypothesis. The irreconcilability of p-values and evidence (with discussion). J Am Statist Assoc 1987; 82: 112-139.

Berger JO, Wolpert RL. The Likelihood Principle. IMS, Hayward: CA 1984.

Berger JO, Wolpert RL. The Likelihood Principle, 2nd ed. (with discussion). IMS, Hayward: CA 1988.

Berk RH, Limiting behavior of posterior distributions when the model is incorrect. Ann Math Statist 1966; 37: 51-58.

Bernardo JM, A Bayesian analysis of classical hypothesis testing. Universidad de Valencia 1980; 605-617.

Bernardo JM, Rueda R. Bayesian Hypothesis Testing: A Reference Approach. Int Statist Rev 2002; 1-22.

Braun HI, The Collected Works of John W. Tukey. Vol. VIII: Multiple Comparisons: 1948-1983. New York: Chapman & Hall 1994.

Brownie C, Keifer J. The Ideas of Conditional Confidence in the simplest setting. Comm. Statist. Theory Methods 1977; 6: 691-751.

Cassela G, Berger RL, Reconciling evidence in the one-sided testing problem (with discussion). J Am Statist Assoc 1987; 82: 106-112.

Casella G, Wells MT. Comparing p-values to Neyman-Pearson tests. Technical Report BU-1073-M, Biometrics Unit and Statistics Cent., Cornell Univ 1990.

Christensen R. Testing Fisher, Neyman, Pearson, and Bayes. Am Statist 2005; 59(2): 121-126.

Dass SC, Berger JO. Unified Conditional Frequentist and Bayesian Testing of Composite Hypotheses. Scand J Statist 2003; 30(1): 193-210.

Delampady M, Berger JO. Lower bounds on Bayes factors for the multinomial distribution, with application to chi-squared tests of fit. Ann Statist 1990; 18: 1295-1316.

Edwards W, Lindman H, Savage LJ. Bayesian statistical inference for psychological research. Psychol Rev 1963; 70: 193-242.

Efron B. Large-Scale Simultaneous Hypothesis Testing. J Am Statist Assoc 2004; 99(465): 96-104.

Fisher RA. Statistical Methods for Research Workers, London: Oliver and Boyd 1925.

Gómez-Villegas MA, González-Pérez B. A Bayesian Analysis for the Homogeneity Testing Problem Using -Contaminated Priors. Commun Statist - Theory Methods 2011; 40(6): 1049-1062.

Gómez-Villegas MA, Maín P, Sanz L. A Bayesian analysis for the multivariate point null testing problem. Statistics 2009; 43(4): 379-391.

Good IJ. The Bayesian/Non-Bayesian compromise: a brief review. J Am Statist Assoc 1992; 87: 597-606.

Hochberg Y, Tamhance AC. Multiple Comparison Procedures, New York: Wiley 1987.

Hoppe FM. Multiple Comparisons. Selection and Applications in Biometry, New York: Dekker 1993.

Hsu JC. Multiple Comparisons: Theory and methods, New York: Chapman & Hall 1996.

Hubbard R, Bayarri MJ. Confusion over Measures of Evidence (p’s) Versus Errors (α’s) in Classical Statistical Testing. Am Statist 2003; 57: 171-177.

Hwang JT, Casella G, Robert Ch, Wells MT, Farrell RH. Estimation of Accuracy in Testing. Ann Statist 1992 20(1): 490-509.

Jeffreys H. Theory of Probability, 1st ed. Oxford: The Clarendon Press 1939.

Kachiashvili KJ. Bayesian algorithms of many hypothesis testing, Tbilisi: Ganatleba 1989.

Kachiashvili KJ. Generalization of Bayesian Rule of Many Simple Hypotheses Testing. Int J Informat Technol Decision Making 2003; 2(1): 41-70.

Kachiashvili KJ. Investigation and Computation of Unconditional and Conditional Bayesian Problems of Hypothesis Testing. ARPN J Syst Softw 2011; 1(2): 47-59.

Kachiashvili KJ. The Methods of Sequential Analysis of Bayesian Type for the Multiple Testing Problem. Sequential Anal 2014; 33: 23-38.

Kachiashvili GK, Kachiashvili KJ, Mueed A. Specific Features of Regions of Acceptance of Hypotheses in Conditional Bayesian Problems of Statistical Hypotheses Testing. Sankhya: Ind J Statist 2012; 74(1): 112-125.

Kachiashvili KJ, Hashmi MA. About Using Sequential Analysis Approach for Testing Many Hypotheses. Bulletin of the Georgian Acad Sci 2010; 4(2): 20-25.

Kachiashvili KJ, Hashmi MA, Mueed A. Sensitivity Analysis of Classical and Conditional Bayesian Problems of Many Hypotheses Testing. Communin Statist—Theory Methods 2012; 41(4): 591-605.

Kachiashvili KJ, Mueed A. Conditional Bayesian Task of Testing Many Hypotheses. Statistics 2011; 47(2): 274-293.

Klockars AJ, Sax G. Multiple Comparison. Newbury Park, CA: Sage 1986.

Kiefer J. Conditional confidence statement and confidence estimations (with discussion). J Am Statist Assoc 1977; 72(360): 789-808.

Lehmann EL. The Fisher, Neyman-Pearson Theories of Testing Hypotheses: One Theory or Two? Am Statist Assoc J Theory Methods 1993; 88(424): 1242-1249.

Lehmann E L. Testing Statistical Hypotheses, 2nd ed. New York: Springer 1997.

Miller RG. Simultaneous Statistical Inference, New York: Wiley 1966.

Moreno E, Giron FJ. On the Frequentist and Bayesian Approaches to Hypothesis Testing, SORT 30(1) January-June 2006; 3-28.

Neyman J, Pearson E. On the Use and Interpretation of Certain Test Criteria for Purposes of Statistical Inference. Part I, Biometrica 1928 20A: 175-240.

Neyman J, Pearson E. On the Problem of the Most Efficient Tests of Statistical Hypotheses, Philos. Trans Roy Soc Ser A 1933; 231: 289-337.

Rao CR. Linear Statistical Inference and Its Application, 2nd ed. New York: Wiley 2006.

Rosenthal R, Rubin DB. Psychological Bulletin 1983; 94(3): 540-541.

Sage AP, Melse JL. Estimation Theory with Application to Communication and Control. New York: McGraw-Hill 1972.

Schaarfsma W, Tobloom J, Van der Menlen B. Discussing truth or falsity by computing a q-value. Statistical Data Analysis and Inference, Ed. Y. Dodge, pp. 85-100. Amsterdam: North-Holland 1989.

Shaffer JP. Modified Sequentially Rejective Multiple Procedures. J Am Statist Assoc 1986; 81(395): 826-831.

Shaffer JP. Multiple hypothesis testing. Annu Rev Psychol 1995; 46: 561-84.

Toothaker LE. Multiple Comparisons for Researchers, NewBury Park, CA: Sage 1991.

Wald A. Sequential analysis. New-York: Wiley 1947.

Wald A. Foundations of a General Theory of Sequential Decision Functions. Econometrica 1947; 15: 279-313.

Westfall PH, Johnson WO, Utts JM. A Bayesian Perspective on the Bonferroni Adjustment. Biometrica 1997; 84(2): 419-427.

Westfall PH, Young SS. Resampling-based Multiple Testing, New York: Wiley 1993.

Wolpert RL. Testing simple hypotheses. Data Analysis and Information Systems. Ed. H.H. Bock & W. Polasek, 7, Heidelberg: Springer 1996; pp. 289-297.




How to Cite

Kachiashvili, K. (2014). Comparison of Some Methods of Testing Statistical Hypotheses: (Part I. Parallel Methods). International Journal of Statistics in Medical Research, 3(2), 174–197.



General Articles