Manual for the determination of Effect size indices and Practical Significance
H. S. Steyn (jr)
Preface
In my work as a statistical consultant over the last twenty years I have found that there is a dire need to be able to make assertions concerning practical significance, particularly in Human sciences. This requirement exists for two reasons.
Firstly, large samples typically show statistically significant differences or relationships when these results are not necessarily important. The reason for this lies primarily in the fact that the rejection of null hypotheses gives one cause to accept alternative hypotheses which are based solely on a difference or relationship and do not convey any information concerning the size of the difference or relationship. Effect size indices based on the sample, however, can be used to determine the size of the difference or relationship so that one can make claims regarding their practical significance.
Secondly, one occasionally finds that one works with complete populations. Drawing samples from the population is no longer an issue and so statistical significance is not applicable in these cases, but practical significance is. Effect size indices then serve as important tools in these situations.
To my knowledge, the statistical literature has not encapsulated this topic in a single compilation and so it was felt that there was a need to create a manual to guide the researcher on this subject matter. It is my hope that this manual can help fill this void in the literature. My approach requires that the reader has some basic knowledge of Statistics and, preferably, is able to help themselves when performing simple Statistical analyses using computer packages like EXCEL, STATISTICA, SAS,SPSS, etc. This manual not only provides the principles of effect sizes, but also attempts to illustrate, using examples and real data, how each one can be calculated and interpreted.
Together with the manual, the programs which can be used to calculate certain of the effect sizes are conveniently made available to the reader through the internet. Some of the programs, which I personally wrote in SAS, are also available for use. The manual clearly references all of the available computer programs.
My thanks go out to my employer who provided me the opportunities throughout the years to be able to do this work. Further, all of my clients’ problems served as the stimulus to allow me to understand the importance of effect size indices and to appreciate the application thereof in research. My colleagues at Statistical Consultation Services, Dr. Suria Ellis, Dr. Gerhard Koekemoer, Mrs. Wilma Breytenbach and Prof. Jan du Plessis, who supported me throughout and collaborated with my attempts to propagate and endorse the use of effect size indices everywhere, deserve special mention and thanks. Similarly, I would also like to give recognition to my previous colleagues Mrs. Susan Uys, Mr. Pirow Engelbrecht and Dr. Lienki Viljoen for their contributions. I owe a great deal of thanks to Mrs. Christa Labuschagne for the superb and accurate word processing of the original text; without her the completion this manual would have been very difficult indeed. The English translation of this manual was carried out in a competent and professional manner by my colleague at the Statistics Subject group, Mr. Leonard Santana and has made it more accessible to a larger group of readers. Thanks also go to my wife, Ina, who, as always, has supported me with her reassuring love and encouragement and has enabled me to complete this work efficiently. I acknowledge my dependency on God, my creator, who gives me energy and ability, and without whom none of this would have been possible.
Faans Steyn,
April 2009
Preface at the Second Edition
Some topics were added to the former manual while it was also updated with additional and more recent references. My thanks go out to my employer who provided me with the opportunity to do this revision. My gratitude again to Dr. Leonard Santana for his translation of the additions in English.
The new added topics are:
• Counter-null effect sizes and their interpretations (paragraphs 4.1.3, 4.1.4, 4.5, 5.1.3 en 5.4.3).
• Binomial effect size display (BESD) (paragraph 5.4.2).
• Effect sizes when comparing more than two proportions (paragraph 6.1.3).
• Limits of agreement and reliability coefficients (with relation to Bland-Altman plots) as effect sizes (paragraph 6.2.4).
• Effect sizes when comparing two univariate populations: classification with ROC-analysis (paragraph 8.2.4).
Faans Steyn
February 2012
• Manual