# Comparison of Some Methods of Testing Statistical Hypotheses: (Part I. Parallel Methods)

## Authors

• K.J. Kachiashvili Georgian Technical University, 77, st. Kostava, Tbilisi, 0175, Georgia, I. Vekua Institute of Applied Mathematics of the Tbilisi State University, 2, st. University, Tbilisi, 0179, Georgia

## Keywords:

Hypotheses testing, -value, likelihood ratio, frequentist approaches, Bayesian approach, constrained Bayesian method, decision regions

## Abstract

The article focuses on the discussion of basic approaches to hypotheses testing, which are Fisher, Jeffreys, Neyman, Berger approaches and a new one proposed by the author of this paper and called the constrained Bayesian method (CBM). Wald and Berger sequential tests and the test based on CBM are presented also. The positive and negative aspects of these approaches are considered on the basis of computed examples. Namely, it is shown that CBM has all positive characteristics of the above-listed methods. It is a data-dependent measure like Fisher’s test for making a decision, uses a posteriori probabilities like the Jeffreys test and computes error probabilities Type I and Type II like the Neyman-Pearson’s approach does. Combination of these properties assigns new properties to the decision regions of the offered method. In CBM the observation space contains regions for making the decision and regions for no-making the decision. The regions for no-making the decision are separated into the regions of impossibility of making a decision and the regions of impossibility of making a unique decision. These properties bring the statistical hypotheses testing rule in CBM much closer to the everyday decision-making rule when, at shortage of necessary information, the acceptance of one of made suppositions is not compulsory. Computed practical examples clearly demonstrate high quality and reliability of CBM. In critical situations, when other tests give opposite decisions, it gives the most logical decision. Moreover, for any information on the basis of which the decision is made, the set of error probabilities is defined for which the decision with given reliability is possible.

## Author Biography

### K.J. Kachiashvili, Georgian Technical University, 77, st. Kostava, Tbilisi, 0175, Georgia, I. Vekua Institute of Applied Mathematics of the Tbilisi State University, 2, st. University, Tbilisi, 0179, Georgia

Georgia, I. Vekua Institute of Applied Mathematics of the Tbilisi State University

## References

Bauer P. et al. Multiple Hypothesenprüfung, (Multiple Hypotheses Testing), Berlin: Springer-Verlag (In German and English) 1988. DOI: https://doi.org/10.1007/978-3-642-52307-6

Berger JO. Could Fisher, Jeffreys and Neyman have Agreed on Testing? Statist Sci 2003; 18: 1-32. http://dx.doi.org/10.1214/ss/1056397485 DOI: https://doi.org/10.1214/ss/1056397485

Berger JO, Boukai B, Wang Y. Unified Frequentist and Bayesian Testing of a Precise Hypothesis. Statist Sci 1997; 12(3): 133-160. DOI: https://doi.org/10.1214/ss/1030037904

Berger JO, Boukai B, Wang Y. Simultaneous Bayesian-frequentist sequential testing of nested hypotheses. Biometrika 1999; 86: 79-92. http://dx.doi.org/10.1093/biomet/86.1.79 DOI: https://doi.org/10.1093/biomet/86.1.79

Berger JO, Brown LD, Wolpert RL. A Unified Conditional Frequentist and Bayesian Test for Fixed and Sequential Simple Hypothesis Testing. Ann Statist 1994; 22(4): 1787-1807. http://dx.doi.org/10.1214/aos/1176325757 DOI: https://doi.org/10.1214/aos/1176325757

Berger JO, Delampady M. Testing precise hypothesis (with discussion). Statist Sci 1987; 2: 317-352. http://dx.doi.org/10.1214/ss/1177013238 DOI: https://doi.org/10.1214/ss/1177013238

Berger JO, Sellke T. Testing a point null hypothesis. The irreconcilability of p-values and evidence (with discussion). J Am Statist Assoc 1987; 82: 112-139. DOI: https://doi.org/10.2307/2289131

Berger JO, Wolpert RL. The Likelihood Principle. IMS, Hayward: CA 1984.

Berger JO, Wolpert RL. The Likelihood Principle, 2nd ed. (with discussion). IMS, Hayward: CA 1988. DOI: https://doi.org/10.1007/978-1-4613-8768-8_15

Berk RH, Limiting behavior of posterior distributions when the model is incorrect. Ann Math Statist 1966; 37: 51-58. http://dx.doi.org/10.1214/aoms/1177699597 DOI: https://doi.org/10.1214/aoms/1177699597

Bernardo JM, A Bayesian analysis of classical hypothesis testing. Universidad de Valencia 1980; 605-617. DOI: https://doi.org/10.1007/BF02888370

Bernardo JM, Rueda R. Bayesian Hypothesis Testing: A Reference Approach. Int Statist Rev 2002; 1-22. DOI: https://doi.org/10.2307/1403862

Braun HI, The Collected Works of John W. Tukey. Vol. VIII: Multiple Comparisons: 1948-1983. New York: Chapman & Hall 1994.

Brownie C, Keifer J. The Ideas of Conditional Confidence in the simplest setting. Comm. Statist. Theory Methods 1977; 6: 691-751. http://dx.doi.org/10.1080/03610927708827526 DOI: https://doi.org/10.1080/03610927708827526

Cassela G, Berger RL, Reconciling evidence in the one-sided testing problem (with discussion). J Am Statist Assoc 1987; 82: 106-112. http://dx.doi.org/10.1080/01621459.1987.10478396 DOI: https://doi.org/10.1080/01621459.1987.10478396

Casella G, Wells MT. Comparing p-values to Neyman-Pearson tests. Technical Report BU-1073-M, Biometrics Unit and Statistics Cent., Cornell Univ 1990.

Christensen R. Testing Fisher, Neyman, Pearson, and Bayes. Am Statist 2005; 59(2): 121-126. http://dx.doi.org/10.1198/000313005X20871 DOI: https://doi.org/10.1198/000313005X20871

Dass SC, Berger JO. Unified Conditional Frequentist and Bayesian Testing of Composite Hypotheses. Scand J Statist 2003; 30(1): 193-210. http://dx.doi.org/10.1111/1467-9469.00326 DOI: https://doi.org/10.1111/1467-9469.00326

Delampady M, Berger JO. Lower bounds on Bayes factors for the multinomial distribution, with application to chi-squared tests of fit. Ann Statist 1990; 18: 1295-1316. http://dx.doi.org/10.1214/aos/1176347750 DOI: https://doi.org/10.1214/aos/1176347750

Edwards W, Lindman H, Savage LJ. Bayesian statistical inference for psychological research. Psychol Rev 1963; 70: 193-242. http://dx.doi.org/10.1037/h0044139 DOI: https://doi.org/10.1037/h0044139

Efron B. Large-Scale Simultaneous Hypothesis Testing. J Am Statist Assoc 2004; 99(465): 96-104. http://dx.doi.org/10.1198/016214504000000089 DOI: https://doi.org/10.1198/016214504000000089

Fisher RA. Statistical Methods for Research Workers, London: Oliver and Boyd 1925.

Gómez-Villegas MA, González-Pérez B. A Bayesian Analysis for the Homogeneity Testing Problem Using -Contaminated Priors. Commun Statist - Theory Methods 2011; 40(6): 1049-1062. http://dx.doi.org/10.1080/03610920903520547 DOI: https://doi.org/10.1080/03610920903520547

Gómez-Villegas MA, Maín P, Sanz L. A Bayesian analysis for the multivariate point null testing problem. Statistics 2009; 43(4): 379-391. http://dx.doi.org/10.1080/02331880802505173 DOI: https://doi.org/10.1080/02331880802505173

Good IJ. The Bayesian/Non-Bayesian compromise: a brief review. J Am Statist Assoc 1992; 87: 597-606. http://dx.doi.org/10.1080/01621459.1992.10475256 DOI: https://doi.org/10.1080/01621459.1992.10475256

Hochberg Y, Tamhance AC. Multiple Comparison Procedures, New York: Wiley 1987. DOI: https://doi.org/10.1002/9780470316672

Hoppe FM. Multiple Comparisons. Selection and Applications in Biometry, New York: Dekker 1993.

Hsu JC. Multiple Comparisons: Theory and methods, New York: Chapman & Hall 1996. http://dx.doi.org/10.1007/978-1-4899-7180-7 DOI: https://doi.org/10.1201/b15074

Hubbard R, Bayarri MJ. Confusion over Measures of Evidence (p’s) Versus Errors (α’s) in Classical Statistical Testing. Am Statist 2003; 57: 171-177. http://dx.doi.org/10.1198/0003130031856 DOI: https://doi.org/10.1198/0003130031856

Hwang JT, Casella G, Robert Ch, Wells MT, Farrell RH. Estimation of Accuracy in Testing. Ann Statist 1992 20(1): 490-509. http://dx.doi.org/10.1214/aos/1176348534 DOI: https://doi.org/10.1214/aos/1176348534

Jeffreys H. Theory of Probability, 1st ed. Oxford: The Clarendon Press 1939.

Kachiashvili KJ. Bayesian algorithms of many hypothesis testing, Tbilisi: Ganatleba 1989.

Kachiashvili KJ. Generalization of Bayesian Rule of Many Simple Hypotheses Testing. Int J Informat Technol Decision Making 2003; 2(1): 41-70. http://dx.doi.org/10.1142/S0219622003000525 DOI: https://doi.org/10.1142/S0219622003000525

Kachiashvili KJ. Investigation and Computation of Unconditional and Conditional Bayesian Problems of Hypothesis Testing. ARPN J Syst Softw 2011; 1(2): 47-59.

Kachiashvili KJ. The Methods of Sequential Analysis of Bayesian Type for the Multiple Testing Problem. Sequential Anal 2014; 33: 23-38. http://dx.doi.org/10.1080/07474946.2013.843318 DOI: https://doi.org/10.1080/07474946.2013.843318

Kachiashvili GK, Kachiashvili KJ, Mueed A. Specific Features of Regions of Acceptance of Hypotheses in Conditional Bayesian Problems of Statistical Hypotheses Testing. Sankhya: Ind J Statist 2012; 74(1): 112-125. DOI: https://doi.org/10.1007/s13171-012-0014-8

Kachiashvili KJ, Hashmi MA. About Using Sequential Analysis Approach for Testing Many Hypotheses. Bulletin of the Georgian Acad Sci 2010; 4(2): 20-25.

Kachiashvili KJ, Hashmi MA, Mueed A. Sensitivity Analysis of Classical and Conditional Bayesian Problems of Many Hypotheses Testing. Communin Statist—Theory Methods 2012; 41(4): 591-605. http://dx.doi.org/10.1080/03610926.2010.510255 DOI: https://doi.org/10.1080/03610926.2010.510255

Kachiashvili KJ, Mueed A. Conditional Bayesian Task of Testing Many Hypotheses. Statistics 2011; 47(2): 274-293. http://dx.doi.org/10.1080/02331888.2011.602681 DOI: https://doi.org/10.1080/02331888.2011.602681

Klockars AJ, Sax G. Multiple Comparison. Newbury Park, CA: Sage 1986. DOI: https://doi.org/10.4135/9781412985185

Kiefer J. Conditional confidence statement and confidence estimations (with discussion). J Am Statist Assoc 1977; 72(360): 789-808. http://dx.doi.org/10.2307/2286460 DOI: https://doi.org/10.1080/01621459.1977.10479956

Lehmann EL. The Fisher, Neyman-Pearson Theories of Testing Hypotheses: One Theory or Two? Am Statist Assoc J Theory Methods 1993; 88(424): 1242-1249. DOI: https://doi.org/10.1080/01621459.1993.10476404

Lehmann E L. Testing Statistical Hypotheses, 2nd ed. New York: Springer 1997. DOI: https://doi.org/10.1214/ss/1029963261

Miller RG. Simultaneous Statistical Inference, New York: Wiley 1966.

Moreno E, Giron FJ. On the Frequentist and Bayesian Approaches to Hypothesis Testing, SORT 30(1) January-June 2006; 3-28.

Neyman J, Pearson E. On the Use and Interpretation of Certain Test Criteria for Purposes of Statistical Inference. Part I, Biometrica 1928 20A: 175-240. DOI: https://doi.org/10.1093/biomet/20A.1-2.175

Neyman J, Pearson E. On the Problem of the Most Efficient Tests of Statistical Hypotheses, Philos. Trans Roy Soc Ser A 1933; 231: 289-337. DOI: https://doi.org/10.1098/rsta.1933.0009

Rao CR. Linear Statistical Inference and Its Application, 2nd ed. New York: Wiley 2006.

Rosenthal R, Rubin DB. Psychological Bulletin 1983; 94(3): 540-541. http://dx.doi.org/10.1037/0033-2909.94.3.540 DOI: https://doi.org/10.1037/0033-2909.94.3.540

Sage AP, Melse JL. Estimation Theory with Application to Communication and Control. New York: McGraw-Hill 1972.

Schaarfsma W, Tobloom J, Van der Menlen B. Discussing truth or falsity by computing a q-value. Statistical Data Analysis and Inference, Ed. Y. Dodge, pp. 85-100. Amsterdam: North-Holland 1989. DOI: https://doi.org/10.1016/B978-0-444-88029-1.50013-7

Shaffer JP. Modified Sequentially Rejective Multiple Procedures. J Am Statist Assoc 1986; 81(395): 826-831. http://dx.doi.org/10.1080/01621459.1986.10478341 DOI: https://doi.org/10.1080/01621459.1986.10478341

Shaffer JP. Multiple hypothesis testing. Annu Rev Psychol 1995; 46: 561-84. http://dx.doi.org/10.1146/annurev.ps.46.020195.003021 DOI: https://doi.org/10.1146/annurev.ps.46.020195.003021

Toothaker LE. Multiple Comparisons for Researchers, NewBury Park, CA: Sage 1991.

Wald A. Sequential analysis. New-York: Wiley 1947.

Wald A. Foundations of a General Theory of Sequential Decision Functions. Econometrica 1947; 15: 279-313. http://dx.doi.org/10.2307/1905331 DOI: https://doi.org/10.2307/1905331

Westfall PH, Johnson WO, Utts JM. A Bayesian Perspective on the Bonferroni Adjustment. Biometrica 1997; 84(2): 419-427. http://dx.doi.org/10.1093/biomet/84.2.419 DOI: https://doi.org/10.1093/biomet/84.2.419

Westfall PH, Young SS. Resampling-based Multiple Testing, New York: Wiley 1993.

Wolpert RL. Testing simple hypotheses. Data Analysis and Information Systems. Ed. H.H. Bock & W. Polasek, 7, Heidelberg: Springer 1996; pp. 289-297. DOI: https://doi.org/10.1007/978-3-642-80098-6_24

2014-05-14

## How to Cite

Kachiashvili, K. (2014). Comparison of Some Methods of Testing Statistical Hypotheses: (Part I. Parallel Methods). International Journal of Statistics in Medical Research, 3(2), 174–197. https://doi.org/10.6000/1929-6029.2014.03.02.11

General Articles