Dr. Alan Reifman's Intro Stats Page

2011-08-24T16:39:00.000-07:00

Welcome to QM I for Fall 2011. In anticipation of tomorrow's opening class, I've collected links for my lecture notes on the topics we will cover (asterisked [*] pages are from my undergraduate research-methods class). These are shown below:

Introductory ideas (here and here)

Units of analysis*

Sampling*

Types of Measures*

Percentiles, histograms, and distributions

Descriptive statistics:* Central tendency (mean, median, and mode) and spread (standard deviation); moments of a distribution; and z-scores (here, here, and here)

Probability (here and here)

Correlation and significance-testing

t-tests

Chi-square

Non-parametric statistics

Statistical power

Confidence intervals

2011-03-06T21:31:00.000-08:00

This year, I've tried to compile my annual list of summer statistics and methodology programs in tabular form (HTML table formatting, though not exactly welcoming, didn't look prohibitively difficult!). This format should allow readers to see different programs' deadline and session dates at a glance. Also, I'm creating separate tables for the U.S./North America and Europe. As always, please notify me (via the link to my faculty webpage in the right-hand column) of errors, omissions, bad links, etc.

UNITED STATES/NORTH AMERICA

Location (link)	Deadline	Sessions	Topics
APA Advanced Training Inst.	...	...	...
U Cincinnati	4/6	6/20-24	Nonlinear Methods
Michigan St U	3/30	6/20-24	Diverse Racial/Ethnic Groups
UC Davis	3/30	6/20-24	SEM/Longitudinal
UC Davis	3/30	6/27-7/1	Exploratory Data Mining
American Statistical Assn(Miami Beach)	FCFS	7/30-8/4	Continuing Education Programming at Joint Statistical Meetings
Ctr for the Adv of Res Methods and Analysis (CARMA)	...	...	Ongoing Workshops Around the World, Including 5/16-21 at Virginia Commonwealth Univ.
Center for Spatially Integrated Social Science	3/31	6/19-24 7/10-15	Spatial Regression, Penn State Multilevel Model, UC Santa Barb
Claremont Grad Univ	?	August 19-22	Evaluation and Applied Research Methods
Curran-Bauer Analytics (UNC)	?	5/23-27 6/6-10	SEM Multilevel Linear Models
DATIC (UConn)	All full; waitlist only	6/13-17 6/20-24 6/27-7/1	HLM SEM Dyadic Analysis
Johns Hopkins/ Bloomberg SPH	...	6/13-7/1 (courses of varying length)	Statistical and Epidemiological Techniques, Content Areas (e.g., Tobacco, HIV/AIDS)
Michigan State University Added 6/1	?	7/11-15	Dyadic data analysis
Muthen Mplus Courses	...	...	Short courses through the year at various locations
Oregon St.	?	7/12-15	Multilevel/Longitudinal Models using Stata (first two days); Longitudinal Modeling using Mplus (last two days)
Portland St. (OR)	?	6/13-14 6/15 6/16-17 6/18	Missing Data Power Analysis HLM "R" (and "lessR")
STATA Usage Workshops	...	...	Short courses throughout year in U.S. and other countries
Statistical Horizons	...	...	Brief workshops, covering different topics, offered in Philadelphia and elsewhere
Texas A&M Updated 4/8	4/ 29 (Early-bird)	5/22-27	Intermediate Stats, HLM, Item Response Theory, Meta-Analysis, SEM, Qualitative Analysis
U at Buffalo	?	5/12-13 5/16-20 5/23-27	Social Networks SEM HLM
U Illinois, Urbana-Cham	5/20 (or earlier if filled up)	6/6-9 6/10	Model Based Measurement and Computerized Adaptive Testing Mixed Methods
U Kansas "Stats Camp"	Different for CE credits, student awards...	6/6-24	Nine courses, each five days, including SEM (basic and long.), multilevel, meta-analysis, categorical, "R," testing, networks, missing data
U Kentucky "LINKS Ctr"	5/15	6/5-10	Social Network Analysis
UMass Amherst	?	6/1-3 6/6-8 6/13-17 6/28-7/1	Social Network Analysis Developmental Trajectories HLM Diary Data/HLM
U Michigan Ann Arbor (ICPSR)	5/1*	6/20-7/15, 7/18-8/12 & shorter	Intro, Regression, Multivariate, SEM, Bayes, Network Analysis, Longitudinal, Latent Class, "R," ...
U Michigan Ann Arbor (SPH)	6/1	7/10-29 (1 & 3 wk courses)	Biostat, Epi, Logistic Regression, Meta-Analysis, Clinical Trials, Survival Analysis, Content Areas (e.g., Cancer, Infectious, Genetic, Injury/Violence)...
U Michigan Ann Arbor (SRC)	5/23	6/6-7/1, 7/5-7/29 (+ 1, 2, 8 wks)	Questionnaire Design, Sampling, Data Analysis, Qualitative, Focus Groups, Interviewing, Program Evaluation, Longitudinal...
U Minnesota (SPH) Added 3/15	?	5/23-6/10	Mainly public health content courses, but some methods courses (e.g., GIS, participatory epidem., qualitative analysis, secondary/census data)
U Washington (SISMID) Added 3/24	5/16 (Early-bird)	6/13-29	Statistical/methodological and public-health content courses (many w/ mathematical modeling)

*Deadline date to receive discount; FCFS = First Come, First Served

EUROPE

Location(link)	Deadline	Sessions	Topics
Ctr for the Adv of Res Methods and Analysis (CARMA)	...	...	Ongoing Series of Workshops Around the World
Essex UK	6/1	7/11-22 7/25-8/5 8/8-19	Numerous statistical, methodological, and applied courses
European Ed. Prog. in Epi. (Florence, It)	5/6	6-20-24 6/27-7/15	Genetic Epidem. Epidemiology
London School of Economics	...	7/25-8/12	Survey Methods and Analysis
National Centre for Research Methods (UK)	...	...	Ongoing courses (including brief ones)
Oslo (Norway) Summer School	?	7/25-29 8/1-5	Mostly Political Science courses, but also specialized methods courses
Modern Methods in Biostat & Epi (Parma, Italy)	5/31	6/5 6/6-11 6/6-11 6-12 6-13/18 6/13-18	Stata: Intro,Meta-Analysis,Tables Biostat, Regression, Survival Epi, Causal, Logistic Regression Stata: Intro, Survival, Missing Epi, Longitudinal, Health Eval Biostat, Clin Trials, Ev-Based
Muthen Mplus Courses	...	...	Short courses through the year at various locations
U Calabria (SDIPA) Italy	?	7/18-22 7/25-29	SEM Experimental Design
U Sheffield UK "Figure It Out" Added 4/8	?	6/29 6/30 7/1	Data Mgt/SPSS Multilevel Modeling/SPSS SEM/Mplus
U Ulster (N. Ireland UK)Added 7/22	Contact: Sharon Adams lifelonglearning @ulster.ac.uk	8/22-9/2	Regression, Factor Analysis, General Linear Model, SEM, Latent Growth Models, Latent Class Analysis, and Multilevel Modelling
XL Stat (Paris)Added 7/11	?	9/26-28	Partial Least Squares

OTHER LOCATIONS

Location (link)	Deadline	Sessions	Topics
Hyderabad (India, IBS) Added 4/21	By 5/20 (10% off) or 5/30	6/2-4	SEM

2010-11-23T09:38:00.000-08:00

Today, we'll be covering confidence intervals (CI). We've alluded to the basic idea in terms of margin-of-error in a political poll, but today, we'll have a more formal treatment of CI's. Many researchers feel CI's are the best way to present results, rather than null-hypothesis significance testing. CI's have taken hold in many fields, but are only slowly catching on in the social sciences. Here are links to my previous postings on the topic.

Primary lecture notes on confidence intervals

General formula for confidence intervals and a song

2010-11-16T09:17:00.000-08:00

This week we'll be covering statistical power (also known as power analysis). Power is not a statistical technique like correlation, t-test, and chi-square. Rather, power involves designing your study (particularly getting a large enough sample size) so that you can use correlations, t-tests, etc., more effectively. The core concept of power, like so much else, goes back to the distinction between the population and a sample. When there truly is a basis in the population for rejecting the null hypothesis (e.g., a non-zero correlation, a non-zero difference between means), we want to increase the likelihood that we reject the null from the analysis of our sample. In other words, we want to be able to pronounce a result significant, when warranted. Here are links to my previous entries on statistical power.

Introductory lecture

Why a powerful design is needed: The population may truly have a non-zero correlation, for example, but due to random sampling error, your sample may not; plus, some songs on statistical power!

Remember that there's also the opposite kind of error: The population truly has absolutely no correlation, but again due to random sampling error, you draw a sample that gives the impression of a non-zero correlation.

How to plan a study using power considerations

2010-11-03T20:42:00.000-07:00

Here are direct links to some old chi-square blog postings. This one discusses the reversibility error and how properly to read an SPSS printout of a chi-square analysis. The other one illustrates the null hypothesis for chi-square analyses in terms of equal pie-charts.

The following photo of the board, containing chi-square tips, was added on November 15, 2011 (thanks to Selen).

Plus a song (added November 1, 2011):

One Degree is Free

Lyrics by Alan Reifman

(May be sung to the tune of “Rock and Roll is Free,” Ben Harper)

Look at your, chi-square table,

If it is, 2-by-2,

One cell can be filled freely,

While the others take their cue,

The formula that you can use,

Come on, from the columns, lose one,

And one, as well, from the rows,

Multiply the two, isn’t this fun?

One degree is free, in your table,

With con-tin-gen-cy, in your table,

One degree is free, in your table,

…free in your table,

Say, your table is larger,

Maybe it’s 2-by-4,

Multiply one by three,

3 df are in store,

The df’s are essential,

To check significance,

Go to your chi-square table,

And find the right instance,

Three degrees are free, in your table,

With con-tin-gen-cy, in your table,

Three degrees are free, in your table,

…free in your table,

(Guitar Solo)

2010-10-26T08:24:00.000-07:00

As we cover t-tests, beyond the introductory notes from my "Basic Statistics Page," we'll be looking at older blog posts here and here.

2010-10-06T22:22:00.000-07:00

At our next class (and possibly the one after that), we'll continue to cover correlational analysis, which we began to do last Tuesday. There are four basic areas to address:

1. A general introduction to correlation, which is available here amidst my Research Methods lecture notes.

2. Running correlations in PASW/SPSS.

3. Statistical significance and testing the null hypothesis, as applied to correlation. Subthemes within this topic include how sample size affects the ease of getting a statistically significant result (i.e., rejecting the null hypothesis of zero correlation in the full population), and one- vs. two-tailed significance.

4. Partial correlation (i.e., the correlation between two variables, holding constant one or more "lurking" variables).

This previous write-up provides a summary of many of these issues.

UPDATE (10/6/11): Here is some new information I've added. First, this graphic tries to make clear that a sample correlation and its significance/probability level are two different things (although related to each other).

Second, in graphing the data points and best-fitting line, you start in "Graphs," go to "Legacy Dialogs," and select "Scatter/Dot." Then, select "Simple Scatter" and click on "Define." You will then insert the variables you want to display on the X and Y axes, and say "OK." When the scatter plot first appears, you can click on it to do more editing. To add the best-fit line, under "Elements," choose "Fit Line at Total."

Initially the dots will all look the same throughout the scatter plot. To make each dot represent the number of cases at that point (either by thickness of the dot or through color-coding), click on the "Binning" icon (circled below in red). Thanks to Xiaohui for finding this!

2010-08-31T09:46:00.001-07:00

Here is the calendar portion of our Fall 2010 syllabus...

2010-03-06T16:45:00.000-08:00

Here is my annual (2010) compendium of statistics and methodology workshops being offered this upcoming summer (primarily in the U.S., but also in Europe). Although details vary by program, they are generally open to graduate students, post-docs, and faculty.

APA Advanced Training Institutes (LINK):

*Structural Equation Modeling in Longitudinal Research, University of Virginia, Charlottesville, May 25-29

*Exploratory Data Mining in Behavioral Research, University of Southern California, June 14-18

*Nonlinear Methods for Psychological Science, University of Cincinnati, June 21-25

*Research Methods with Diverse Racial and Ethnic Groups, Michigan State University, East Lansing, June 21-25, 2010

American Statistical Association/Joint Statistical Meetings (Continuing Education).Vancouver, BC, Canada, July 31-August 5 (LINK, added March 15).

Calabria, Italy (Public Administration, SDIPA). Five-day workshops throughout the summer, on topics such as SEM, network analysis, and research methods (FLYER mostly in Italian; a form entirely in English is available upon request from Nino Miceli at miceli@unical.it; added April 9)

Claremont Graduate University (School of Behavioral and Organizational Sciences). One-day workshops (both on-site and online) focusing on program evaluation, but also covering additional topics in research methods and statistics (LINK; added May 8).

Curran-Bauer Analytics (North Carolina). Multilevel Linear Models, May 24-28, and SEM, June 7-11. University of North Carolina at Chapel Hill (LINK).

Data Analysis Training Institute of Connecticut. HLM "A" June 14-18, 2010; SEM June 14-18 (sold out, waitlist being compiled); HLM "B" June 21-25; Dyadic Data Analysis June 21-25 (sold out, waitlist being compiled) (LINK).

Essex Summer School in Social Science Data Analysis. One- and two-week courses on dozens of topics, July 12-August 20 (LINK; added March 15).

European Educational Programme in Epidemiology. Florence, Italy, June 21-July 9, "The main three week course offers in the first two weeks five general modules on epidemiological study design and statistical analysis of epidemiological data. In the third week six special modules, ranging from cancer epidemiology and fertility and pregnancy to the impact of changes of global climatic environment cover topics of current relevance for health (students can choose which modules to follow)." (LINK; added March 15).

Johns Hopkins University (Bloomberg School of Public Health). Summer Institute of Epidemiology and Biostatistics, June 14-July 2 (LINK).

London School of Economics. Survey methodology workshop, second summer term, July 26-August 13 (LINK; added April 19).

Muthen M-plus Short Courses. Baltimore and Barcelona (LINK).

National Centre for Research Methods (United Kingdom). "The 4th ESRC Research Methods Festival, organised by the National Centre for Research Methods (NCRM), will be held on 5-8 July 2010 at St Catherine's College, Oxford, UK" (LINK).

Norwegian School of Management, Oslo (Center for Applied Statistics).Two-day workshop with Ken Bollen on Longitudinal Modeling with Structural Equation Models, May 31-June 1 (LINK; added April 14).

Oregon State University. Summer Institute on Research Methodology, July 13-15, "2 day workshop including Introduction to Mplus and Latent Growth Curve Modeling using Mplus; one day workshop on Multilevel & Longitudinal Models using Stata" (LINK).

Oslo Summer School in Comparative Social Science Studies. August 2-6, Event History Analysis and the Life Course (LINK; added March 15).

Penn State (Methodology Center). Analysis of Longitudinal Dyadic Data, June 7-9 (LINK; added March 22)

Portland State University (Oregon). A series of two-day workshops in mid-June, on numerous topics (LINK).

Scientific Software International (SSI). Short workshops on SEM/LISREL and HLM in June and September, Chicago (LINK; see "Workshops" heading in left-hand column).

STATA Short Courses. Held all over the world, at many different times of the year, often in three-day workshops (LINK, added March 15).

Statistical Horizons. Brief workshops, covering different topics, offered in Philadelphia and elsewhere (LINK).

Summer School on Modern Methods in Biostatistics and Epidemiology. June 13-26, Cison di Valmarino, Treviso, Italy (LINK; added March 15).

Texas A&M (Educational Psychology). One-day (May 23) meta-analysis workshop; five-day workshops (May 24-28) on other topics (March 31 early-bird registration discount) (LINK).

Universitat Pompeu Fabra (Barcelona). One- and two-day workshops on R statistical package, multilevel models for longitudinal data, and SEM (EQS), from June 28-July 2 (LINK; added May 18).

University at Buffalo (Sociology). Workshops of one week or shorter in May, covering SEM, HLM, and secondary analysis (LINK).

University of California, Davis (Longitudinal Research Institute). Variety of topics in longitudinal research, August 16-18 (LINK; added June 20).

University of Denver (Society for Social Work and Research Summer Quantitative Training Institutes). Social-network analysis (July 21-23) and propensity scores (July 28-30). (LINK; added June 2).

University of Illinois (Educational Psychology). Five-day and shorter workshops in June, covering multilevel modeling, mixed methods, and testing/measurement (LINK; added March 15).

University of Kansas. KU's "Stats Camp" offers several five-day workshops throughout June on numerous topics (LINK).

University of Kentucky (LINKS Center). Brief workshops on various topics pertaining to social network analysis, held from June 7-12 (LINK).

University of Maryland (Center for Integrated Latent Variable Research). Conference on "Advances in Longitudinal Methods in the Social and Behavioral Sciences," June 17-18, with June 16 all-day pre-conference short course (LINK).

University of Massachusetts-Amherst (Center for Research on Families). Five-day workshops throughout June (LINK).

University of Michigan...

ICPSR Quantitative Methods

Survey Research Center

School of Public Health -- Epidemiology

University of New Hampshire (Carsey Institute). "Using Multilevel Modeling to Analyze Longitudinal Data," June 7-9 (LINK; added May 8).

University of Texas, Austin. Wide variety of statistical topics, May 24-27 (priority given to people associated with UT) (LINK; added May 8).

Virginia Commonwealth University (CARMA). Three-day courses on a wide range of topics, offered both in Richmond, VA, and Detroit (Wayne State) (LINK).

2009-10-22T10:34:00.000-07:00

I have just created a new graphic on how to interpret SPSS print-outs for the Independent-Samples t-test.

This new chart supplements the t-test lecture notes I showed yesterday, reflecting a change in my thinking about what to take from the SPSS print-outs.

One of the traditional assumptions of an Independent-Samples t-test is that, before we can test whether the difference between the two groups' means on the dependent variable is significant (which is of primary interest), we must verify that the groups have similar variances (standard-deviations squared) on the DV. This assumption, which is known as homoscedasticity, basically says that we want some comparability to the two groups' distributions in terms of their being equally spread out, before we can compare their means (you might think of this as "outlier protection insurance," although I don't know if this is technically the correct characterization of the problem).

If the homoscedasticity (equal-spread) assumption is violated, all is not lost. SPSS provides a test for possible violation of this assumption (the Levene's test) and, if violated, an alternative solution to use for the t-test. The alternative t-test (known as the Welch t-test or t') corrects for violations of the equal-spread assumption by "penalizing" the researcher with a reduction of degrees of freedom. Fewer degrees of freedom, of course, make it harder to achieve a statistically significant result, because the threshold t-value to attain significance is higher.

Years ago, I created a graphic for how to interpret the Levene's test and implement the proper t-test solution (i.e., the one for equal variances or for unequal variances, as appropriate). Even with the graphic, however, students still found the output confusing. Stemming from these student difficulties and some literature of which I have become aware, I have changed my opinion.

I now subscribe to the opinion of Glass and Hopkins (1996) that, “We prefer the t’ in all situations” (p. 305, footnote 30). Always using the t-test solution for when the two groups are assumed to have unequal spread (as depicted in the top graphic) is advantageous for a few reasons.

It is simpler to always use one solution than go through what many students find to be a cumbersome process for selecting which solution to use. Also, despite the two solutions (assuming equal spread and not assuming equal spread) being different and having different formulas in part, the bottom-line conclusion one draws (e.g., that men drink significantly more frequently than do women) often is the same under both solutions. If anything, the preferred (not assuming equal spread) solution is a little more conservative; in other words, it makes it a little harder to obtain a significant difference between means than does the equal-spread solution. As a result, our findings will have to be a little stronger for us to claim significance, which is not a bad thing.

Lastly, some additional notes: In searching the web to prepare this write-up, I discovered a neat graphical website that lets you see how the t-distribution begins to converge with the z-distribution as the t-test is specified with increasing degrees of freedom. Here's another good all-around site regarding t-tests, from which I learned about the distribution graphic.

Glass, G. V., & Hopkins, K. D. (1996). Statistical methods in psychology and education (3rd ed.). Needham Heights, MA: Allyn & Bacon.

2009-10-09T20:37:00.000-07:00

Here's a photo of the board from today's class, covering correlation and significance testing (thanks to Kristina for the photo). I've annotated the information with some additional clarification.

2009-09-26T12:12:00.000-07:00

Here are a couple of photos of the board (thanks to Kristina) from our coverage of z-scores during recent class meetings. An older photo that's also important can be accessed by clicking here.

What we did in the photo immediately above is examine each year's Texas Tech starting quarterback in terms of where he stood in relation to his Big 12 quarterbacking peers from the same year in average passing yards per game. The bigger the z-score, the better the Tech QB was relative to his same-year peers.

The Big 12 football statistics are available here. If statistics for just Big 12 conference games were available, we used those, as it would help hold the quality of opposition constant. If not, we used statistics from all of a team's cames (including non-conference). To take a concrete example, let's look at 2008 (statistics other than average passing yards per game have been edited out):

PASSING AVG/GAME........Team....Class...Games...Avg/Game
-----------------------------------------------------------------------------
1. Harrell, Graham............. TTU..... SR..... 8... 396.8
2. Bradford, Sam............... OU...... SO..... 8... 348.4
3. Daniel, Chase................. MU...... SR..... 8... 308.5
4. McCoy, Colt................. UT........ JR...... 8... 303.4
5. Ganz, Joe...................... NU........ SR..... 8... 291.9
6. Reesing, Todd............... KU....... JR..... 8... 271.2
7. Arnaud,Austen............... ISU.... SO...... 8... 268.6
8. Johnson, Jerrod............. TAMU.. SO.... 8... 247.9
9. Robinson, Zac................ OSU... JR...... 8... 235.8
10.Freeman, Josh............... KSU... JR...... 8... 230.0
11.Griffin, Robert............... BU.... FR....... 8... 166.9
12.Hawkins, Cody............... CU.... SO...... 8... 135.5

The mean and standard deviation of the 12 numbers in the last column were obtained (the class was broken up into three-person groups, with each group getting a different year). We would take Harrell's value (396.8), subtract the mean (267.1), then divide the result by the sample SD (72.1). The result is Harrell's 2008 z-score, 1.79 (or perhaps 1.80, however one handles rounding issues). The mean and SD of a distribution can conveniently be obtained at an online calculating site called Statiscope.

As we discussed in class, Harrell's 2008 z-score was not as much above the league average as were Harrell's 2006 and 2007 z-scores and other Tech quarterbacks' z-scores for their respective years. In no way does this suggest that Harrell had a subpar year in '08. Rather, the Big 12 had a number of great quarterbacks, which made it harder for Harrell to distance himself from the others. Harrell's QB counterparts included Oklahoma's Sam Bradford and Texas's Colt McCoy, who finished first and second, respectively, in the balloting for the Heisman Trophy, the award given to the finest college football player in the nation.

2009-08-27T14:59:00.000-07:00

Welcome back, Fall 2009 students! Here is the calendar portion of our course syllabus...

2009-03-04T09:47:00.000-08:00

If it's March, it must be time for our annual compendium of statistics and methodology workshops being offered this upcoming summer (focusing primarily on the U.S.). Although details vary by program, they are generally open to graduate students, post-docs, and faculty. Direct quotations in the text below come from messages sent by program organizers to listservs or from program websites.

APA Advanced Training Institutes (LINK):

Ø Non-Linear Methods for Psychological Science (June 8-12, Univ. of Cincinnati)

Ø Research Methods with Diverse Racial & Ethnic Groups (June 22-26, Michigan State Univ.)

Ø Structural Equation Modeling in Longitudinal Research (June 29-July 1, Univ. of Virginia)

Ø Exploratory Data Mining in Behavioral Research (July 20-24, Univ. of Southern California)

Claremont Graduate University. August 21-26. Several workshops, with a focus on program evaluation and applied/social issues (LINK). Updated May 5, 2009

Curran-Bauer Analytics (North Carolina). Multilevel Linear Models, June 1-5, Rizzo Conference Center, University of North Carolina at Chapel Hill (LINK).

Data Analysis Training Institute of Connecticut. HLM, SEM, and Dyadic Analysis will be offered in week-long workshops ranging from June 8-26, taught by David Kenny (about whom I've written a song), Betsy McCoach, and others (LINK).

Johns Hopkins University (Bloomberg School of Public Health). Summer Institute Courses in Mental Health Research, June 22-July 2. Features courses in epidemiology, prevention, policy, grant-writing, randomized trials, longitudinal latent-variable analysis, and missing data (LINK). Added March 9, 2009

Muthen M-plus Short Courses. Held at Johns Hopkins University and other locations (LINK).

National Centre for Research Methods (United Kingdom). See under "Training & Events" on the group's website (LINK).

Oregon State University. July 7-10, on Latent Growth Modeling and M-plus (LINK).

Portland State University (Oregon). A series of two-day workshops in mid-June, on topics such as factor analysis/SEM, longitudinal analysis (including survival analysis), secondary data, and complex survey design (LINK). Added March 25, 2009

Scientific Software International (SSI). Workshops on SEM/LISREL and HLM, September 1-3, Chicago (LINK). Added April 21, 2009

Statistical Horizons. Brief workshops, covering different topics, offered in Philadelphia and elsewhere (LINK).

Texas A&M (Educational Psychology). A wide variety of courses will be offered from June 7-19. "This year we are very excited to have two more new courses: Mediation and Moderation Analysis by Matt Fritz (Virginia Tech) [and] Meta Analysis by Victor Willson (Texas A&M University)." Existing courses include Mixed Methods Research, Item Response Theory, Exploratory and Confirmatory Factor Analysis, SEM, HLM, and Nonparametric Statistics. Discounted "Early Bird" registration by March 31 (LINK).

University at Buffalo (Sociology). Workshops of one week or shorter in May, covering SEM, HLM, meta-analysis, and evaluation (LINK).

University of Kansas. "... is pleased to announce an expanded set of 5-day workshops on quantitative methodology, to take place June 1-19, 2009 in Lawrence, Kansas. Workshop topics this year include: Structural Equation Modeling, Multilevel Modeling, Advanced Longitudinal Modeling, Meta-Analysis, Item Response Theory, and Social Network Dynamics" (LINK).

University of Maryland (Center for Integrated Latent Variable Research). Diagnostic Measurement: Theory, Methods, and Applications, May 28-29. "This short course is targeted to measurement professionals with at least basic training in statistics and who are interested in learning about the theory, methods, and applications of modern latent-variable models for classifying respondents. The models that are the focus of this course are known as diagnostic classification models (DCMs) or, alternatively, cognitive diagnosis models and restricted latent class models..." (LINK).

University of Massachusetts-Amherst (Center for Research on Families). Workshops entitled "Modeling Diary and Dyadic Data" will be held June 8-11 (LINK).

University of Michigan...

Ø ICPSR Quantitative Methods

Ø Survey Research Center

Ø School of Public Health -- Epidemiology

Virginia Commonwealth University (CARMA). Brief courses on a wide range of data-analytic topics, both quantitative and qualitative, offered from May 11-16 (LINK).

2008-11-20T20:32:00.000-08:00

(Updated after Friday, November 21's class)

Friday we'll be covering Confidence Intervals (CI), focusing on my notes from two years ago.

This year, I'd like to add some links to sites on the calculation of different types of CI's. The general form for calculating CI's is:

95% CI = Sample estimate +/- (1.96) (Standard Error)
............(e.g., r or Mean)

The specific forms of this calculation for CI's around a mean, a correlation, and a proportion, respectively, are shown here, here, and here. Note how increasing one's sample size (N) will shrink the SE and hence, the CI. I also have written a new song:

True Value
Lyrics by Alan Reifman
(May be sung to the tune of “Moon Shadow,” Cat Stevens)

Within your CI, you get the true value, true value, true value,
With 95%, you get the true value, true value, true value,

You get a sample statistic, a sample r, or sample M,
You then take plus-or-minus two (it’s really 1.96…), standard errors beyond your stat,
And within this new interval, we can be, so confident,
That the true value, mu or rho, will be somewhere… inside…, our confidence interval,

Within your CI, you get the true value, true value, true value,
With 95%, you get the true value, true value, true value,

2008-11-05T10:24:00.000-08:00

Here are some quick links to the main chi-square notes (here and here). Also, the exit polls from last night's election are available in such a format that we can apply chi-square analysis to them.

2008-10-24T13:48:00.000-07:00

This article on polling in the presidential election mentions a number of concepts that we've covered in our course.

2008-10-22T13:48:00.000-07:00

We will now be covering t-tests (for comparing the means of two groups) for the next week or so. As we'll discuss, there are two ways to design studies for a t-test:

INDEPENDENT SAMPLES (covered in Chapter 14 of the book), where a participant in one group (Obama voters, say) cannot be in the other group (McCain voters).

CORRELATED GROUPS (covered in Chapter 15), where the same (or matched) person(s) can serve in both groups. For example, the same participant could be asked to complete math problems both during a period where loud hard-rock music is played and during a period where quiet, soothing music is played. Or, if you were comparing men and women on some attitude measure and your participants were heterosexual married couples, that would be considered a correlated design.

Because the formula for an independent-samples t-test is somewhat difficult to glean from the textbook (you could start at Equation 14.6 and then fill in the denominator with help from Equation 14.2b), here's a simplified graphic I found from the web (original source):

(NOTE: I have edited this page to reflect changes I have made in referring students to different graphics.)

2008-10-02T10:46:00.001-07:00

The correlation statistic presents the first instance in which we'll be examining statistical significance. The question is whether we can reject the null hypothesis (Ho) that the correlation between a given pair of variables in the full population is zero (RHO = 0).

We, of course, obtain correlations (r) for our sample, and then see if our sample correlation is sufficiently different from zero (in either a positive or negative direction) so that it would have been sufficiently unlikely to have arisen from pure chance when the population RHO was truly zero. Under that situation, we can reject the Ho of zero RHO.

This article by Jerry Dallal provides a concise historical summary of why p < .05 has become the "magic" cutoff point.

Some of my postings from recent years (here and here) address statistical significance with correlations in additional ways.

2008-09-26T20:10:00.000-07:00

Today, as one of the final examples of probability, we discussed the famous Monty Hall Problem (named after the host of the old game show "Let's Make a Deal"), which is described in detail here. To my mind, the clearest explanation of the surprising solution is that given by Leonard Mlodinow's book The Drunkard's Walk. Based on Mlodinow's writing, here's a diagram we created in class (thanks to Kristina for taking the picture):

The basic idea is that scenarios in which switching helps occur twice as often as ones in which switching hurts.

UPDATE (Dec. 19, 2011): I learned of a Mythbusters TV episode, in which the Monty Hall Problem is demonstrated (thanks to Kaitlin Leckie). Here is a video clip; the most useful part, in my view, starts at the 6:30 point.

2008-09-23T12:36:00.000-07:00

I have now corroborated the results of our Michael Phelps/Mark Spitz Olympic swimming z-score class exercise, which I'll show below. This activity was inspired by an earlier study published in the Baseball Research Journal that used z-scores to compare home-run sluggers of different eras.

I shared the activity with two listserve discussion groups, those of the APA Division of Evaluation, Measurement, and Statistics and the Society for Personality and Social Psychology, offering to provide the raw data and documentation on how to conduct the exercise. I'm pleased to report that over 100 people have requested these materials to use in their own statistics classes. The materials can still be requested, via my faculty webpage (see link in the right-hand column). I framed the exercise as follows, in the documentation:

Michael Phelps, with eight gold medals in the 2008 Beijing Olympics (on top of six golds from the 2004 Athens games), and Mark Spitz, with seven gold medals in the 1972 Munich Olympics, are swimming’s two greatest champions.

The two swam many of the same events. Though the respective times by Phelps are several seconds faster than Spitz’s, the 36 years between 1972 and 2008 are a long time for improvements in training, technique, nutrition, and facilities. A statistic known as the z-score allows us to see which swimmer was more dominant relative to his contemporary peers.

Phelps and Spitz had three individual (non-relay) events in common, the 200-meter freestyle, 200-meter butterfly, and the 100-meter butterfly. Because I have a relatively small class and wanted to have groups of three or four students each work on a different segment of the data, we looked at only the first two of the aforementioned events. Two considerations to note are that (a) times were converted into total seconds to facilitate computations; and (b) where an athlete swam multiple races of the same event (i.e., heats, semifinals, and finals), his fastest time was used. Here are the results:

200 FREESTYLE

2008

Mean = 109.42 seconds
SD = 3.20
Phelps time = 102.96 seconds (1:42.96)
Phelps z = -2.02

[For non-statisticians who may be reading this, z = an individual's value minus the mean, with the difference then divided by the standard deviation. The latter represents how spread out the data are.]

1972

Mean = 120.33 seconds
SD = 4.57 seconds
Spitz time = 112.78 seconds (1:52.78)
Spitz z = -1.65

200 BUTTERFLY

2008

Mean = 117.85 seconds
SD = 3.01
Phelps time = 112.03 seconds (1:52.03)
Phelps z = -1.93

1972

Mean = 129.51 seconds
SD = 5.32
Spitz time = 120.70 seconds (2:00.70)
Spitz z = -1.66

Note that negatively signed z scores are a "good" thing, indicating by how much Phelps or Spitz was faster (i.e., consuming less time) than his respective competitors. As can be seen, Phelps was more dominating against the 2008 fields of his events, than Spitz was against the 1972 fields. It would also be interesting to look at the 100-meter freestyle, which of course, Phelps won by the narrowest of margins.

I thank Nancy Genero of Wellesley College, a fellow University of Michigan Ph.D., for sharing the results from her class; by comparing our respective data files for possible typographical errors, we were able to reconcile some minor differences. Also, as a technical note, an "outlier" swimmer who had a time of 2:33.75 in the 1972 200 freestyle (when the next slowest time was around 2:13) was excluded. An extreme value would have affected both the mean and SD, of course.

Unlike the above analyses, which used all competitors in an event (regardless of whether they reached the finals or even the semifinals), one could also look exclusively at the finals. To the extent that qualifying rules for the Olympics may have changed between 1972 and 2008, or that other factors were operative, the proportion of weak swimmers (in a world-class context) in the fields might have been different in the two Games, again possibly affecting the z-score results. University of Nevada Reno graduate student Irem Uz indeed analyzed only the finals, and these were his results:

Phelps 200 free z = -1.92
Spitz 200 free z = -1.34
Phelps 200 fly z = -1.62
Spitz 200 fly z = -2.01

Under this method, there's a little redemption for Spitz. Examining the results of Spitz's 200 fly win in 1972, his dominance is clear:

1. Mark Spitz 2:00.70 WR
2. Gary Hall 2:02.86
3. Robin Backhaus 2:03.23
4. Jorge Delgado, Jr. 2:04.60
5. Hans Faßnacht 2:04.69
6. András Hargitay 2:04.69
7. Hartmut Flöckner 2:05.34
8. Folkert Meeuw 2:05.57

The mean was roughly 2:04, putting Spitz 3.30 seconds faster than it. Meanwhile, the extremely tight clustering of the fourth- through eighth-place swimmers served to keep the overall SD small (1.63). The upshot is a very big z for Spitz.

ADDENDA

The Wall Street Journal's "Numbers Guy," Carl Bialik, provided some other types of Phelps-Spitz comparisons as this year's Olympics were going on.

The New York Times created an amazing slide show of graphics, showing how the swimming times of Phelps and Spitz stacked up against each other, and also how each fared against his respective competition.

Another blogger, Jeremy Yoder, independently came up with the idea to analyze z-scores for Phelps and Spitz. Yoder's results are different from the comparable analyses reported above, for some reason.

2008-09-11T13:37:00.000-07:00

UPDATE: FRIDAY'S CLASSES HAVE BEEN CANCELLED DUE TO THE HEAVY RAINS AND FLOODING

At Friday's class, we'll pick up our discussion of the standard deviation, and how the mean and SD both fit into a larger system of "moments." If we have time, we'll then take up z-scores.

2008-09-07T16:02:00.000-07:00

As we continue with histograms and distributions, this posting of mine from two years ago will guide our discussion. Note: In the newer edition of the textbook, the figure in question is 3.14 on p. 51.

OCTOBER UPDATE: Here are some interesting examples of bimodal distributions.

2008-09-03T11:14:00.000-07:00

For your convenience, here's a copy of the week-to-week schedule (you can click on it to enlarge it)...

2008-08-27T07:26:00.000-07:00

Welcome to introductory graduate statistics in the HDFS department at Texas Tech. My introductory statement from a year ago has some good information, so let's take a look at it.

Also, I recently discovered an older article that sets forth some goals for what you should learn in this class (and other classes).

Utts, J. (2003). What educated citizens should know about statistics and probability. American Statistician, 57(2), 74-79.

We can access this article via the Texas Tech Library website or Google Scholar.

2008-03-10T19:31:00.000-07:00

It's time for our second annual listing of summer statistics and methodology workshops being offered throughout the U.S. Although details vary by program, they are generally open to graduate students, post-docs, and faculty. Direct quotations in the text below come from messages sent by program organizers to listservs or from program websites.

American Psychological Association Advanced Training Institutes. Five specific workshops are offered, as listed below (LINK).

Structural Equation Modeling in Longitudinal Research (June 9-13, Univ. of Virginia)

Non-Linear Methods for Psychological Science (June 9-13, Univ. of Cincinnati)

Research Methods with Diverse Racial & Ethnic Groups (June 23-27, Michigan State Univ.)

Geographic Information Systems for Behavioral Research (July 16-18, Univ. of California, Santa Barbara)

Using Large-Scale Databases: NICHD Study of Early Child Care and Youth Development (Aug. 4-8, Univ. of North Carolina)

Claremont Graduate University (School of Behavioral and Organizational Sciences). A total of 24 workshops, held from August 22-27, 2008. Topics include a heavy emphasis on program evaluation, with qualitative research covered as well as quantitative (LINK). Added June 3, 2008

Georgia State University (College of Business).PLS Path Modeling, a variant of SEM, to be held in Buckhead section of metro Atlanta, June 26-27 (LINK). Added April 13, 2008

Johns Hopkins University (Muthen Mplus courses). August 20, 2008, Introductory and intermediate growth modeling; August 21, 2008, Advanced growth modeling, survival analysis, and missing data analysis; additional courses are offered throughout the year at various locations (LINK, then scroll to bottom; also HERE).

Northwestern University (Institute for Policy Research).
"...Professors Tom Cook of Northwestern University and Will Shadish of University of California, Merced, will be leading two workshops in the summer of 2008 on the design and analysis of practical quasi-experiments for use in education. Workshop dates are August 4-8, and August 11-15" (LINK).

Oregon State University (Summer Institute on Research Methodology). July 8-11, 2008, with a focus on latent growth curve modeling (LINK). Added March 11, 2008

Oxford University, United Kingdom (Research Methods Festival). Short workshops in a large variety of statistical and methodological topics, June 30-July 3 (LINK). Added April 22, 2008

Portland State University (Oregon). Two-day workshops from mid-late June. A mix of popular topics (HLM, SEM) and ones less commonly available, such as survival analysis, and handling missing data (LINK). Added April 4, 2008

Statistical Horizons. Workshops, generally lasting a few days, on a variety of statistical topics; offered in Philadelphia, plus some West Coast and international locations (LINK). Added March 11, 2008

Texas A&M University (Educational Psychology). Several one-week workshops (plus a few half-day ones) in June. Popular techniques such as SEM, HLM, and IRT are offered, along with ones seemingly not as widely available elsewhere, such as effect sizes/confidence intervals and nonparametric statistics (LINK). Added March 17, 2008

University at Buffalo (Sociology). Short workshops (2- and 5-day) from mid-late May, in HLM, SEM, and meta-analysis (LINK). Added April 13, 2008

University of Connecticut (DATIC). "There will be four DATIC workshops offered during June 2008 and August 2008," on HLM, SEM (these two at UConn), and dyadic analysis (at Michigan State). "Each workshop [will] cost $950 if paid by April 1, 2008. Housing can be arranged for each workshop" (LINK).

University of Kansas (KU "Stats Camp"). Five-day workshops in June, covering topics such as SEM, longitudinal analysis, multi-level modeling, meta-analysis, and IRT (LINK). Added March 11, 2008

University of Massachusetts, Amherst (Center for Research on Families). Modeling Diary and Dyadic Data, July 15-17
(LINK). Added June 12, 2008

University of Massachusetts, Amherst (Education). Item Response Theory workshop, Monday, July 28 to Friday, August 1, 2008. Fee $900, includes workshop materials and some meals, but lodging expenses are separate. Limited to 35, first-come first-serve. (Contact Lisa Keller at lkeller@educ.umass.edu for further information). Added March 11, 2008

University of Michigan (ICPSR). From June to August, ICPSR offers four-week courses, as well as 3-5 day workshops (some of the latter are in locations other than Ann Arbor). Most courses are on statistical topics, but others introduce analysis of particular datasets and/or cover thematic content areas in the social sciences (LINK).

University of Michigan (Public Health/Epidemiology). One- and three-week courses in July, on statistical, methodological, and substantive health topics (LINK).

University of Michigan (Survey Research Center). Four-week (and shorter) courses run during June and July on methodological issues in conducting survey research (LINK).

University of North Texas. FAST APPROACHING! May 1-2, Item Response Theory (LINK; for further information, contact Darrell.Hull@unt.edu). Added April 15, 2008

Virginia Commonwealth University (CARMA). Three-day short courses in the middle of May, on a variety of statistical and methodological topics (LINK). Added March 26, 2008

2007-11-28T20:25:00.001-08:00

Below, I've added a new chart, based on things we discussed in today's class. William Trochim's Research Methods Knowledge Base, in discussing statistical power, sample size, effect size, and significance level, notes that, "Given values for any three of these components, it is possible to compute the value of the fourth." The table I've created attempts to convey this fact in graphical form.

You'll notice the (*) notation by "S, M, L" in the chart. Those, of course, stand for small, medium, and large effect sizes. As we discussed in class, Jacob Cohen developed criteria for what magnitude of result constitutes small, medium, and large for correlational studies and those studies comparing means of two groups (t-test type studies, but t itself is not an indicator of effect size).

[Some links have been updated, as of November 18, 2010]

When planning a new study, naturally you cannot know what your effect size will be ahead of time. However, based on your reading of the research literature in your area of study, you should be able to get an idea of whether findings have tended to be small, medium, or large, which you can convert to the relevant values for r or Cohen's d. These, in turn, can be submitted to power-analysis computer programs and online calculators.

I try to err on the side of expecting a small effect size. This will have the effect of requiring me to obtain a large sample size, to be able to detect a small effect, which seems like good practice, anyway.

2007-11-27T18:25:00.000-08:00

Tomorrow, we'll be covering statistical power. Last year's blog notes on this topic are pretty extensive, so I'll just add a few more pieces of information this year (including a couple of songs).

As we'll discuss, the conceptual framework underlying statistical power involves two different kinds of errors: rejecting the null (thus claiming a significant result) when the null hypothesis is really true in the population (known as a "false alarm"); and failing to reject the null when the true population correlation (rho) is actually different from zero (known as a "miss"). The latter is illustrated below:

And here are my two new power-related songs...

Everything’s Coming Up Asterisks
Lyrics by Alan Reifman
(May be sung to the chorus of “Everything’s Coming Up Roses,” from Gypsy, Styne/Sondheim)

State the null! (SLOWLY),
Run the test!
See if H-oh should be, put to rest,

If p’s less, than oh-five,
Then H-oh cannot be, kept alive (SLOWLY),

With small n,
There’s a catch,
There could be findings, you will not snatch,

There’s a chance, you could miss,
Rejecting the null hypothesis… (SLOWLY),

(Bridge)
H-oh testing, how it’s always been done,
Some resisting, will anyone be desisting?

With large n,
You will find,
A problem, of the opposite kind,

Nearly all, you present,
Will be sig-nif-i-cant,

You must start to look more at effect size (SLOWLY),
’Cause, everything’s coming up asterisks, oh-one and oh-five! (SLOWLY)

Believe it or not, the above song has been cited in the social-scientific literature:

Find the Power
Lyrics by Alan Reifman
(May be sung or rapped to the tune of “Fight the Power,” Chuck D/Sadler/Shocklee/ Shocklee, for Public Enemy)

People do their studies, without consideration,
If H-oh, can receive obliteration,
Is your sample large enough?
To find interesting stuff,

P-level and tails, for when H-oh fails,
Point-eight-oh’s the way to go,
That’s the kind of power,
That you’ve got to show,

Look into your mind,
For the effect, you think you’ll find,

Got to put this all together,
Got to build your study right,
Got to give yourself enough, statistical might,

You’ve got to get the sample you need,
Got to learn the way,
Find the power!

Find the power!
Get the sample you need!

Find the power!
Get the sample you need!

2007-11-12T20:51:00.000-08:00

This week, we'll be covering non-parametric (or assumption-free) statistical tests. Parametric techniques, which include the correlation r and the t-test, refer to the use of sample statistics to estimate population parameters (e.g., rho, mu). Thus far, we've come across a number of assumptions that technically are required to be met for doing parametric analyses, although in practice there's some leeway in meeting the assumptions.

Assumptions for parametric analyses, which are also described within the first few pages of Chapter 21, are as follows:

*Data for a given variable are normally distributed in the population.

*Equal-interval measurement.

*Random sampling is used.

*Homogeneity of variance between groups (for t-test).

As discussed in the chapter, one would generally opt for a non-parametric test when there's violation of one or more of the above assumptions and sample size is small. In other words, if assumptions are violated but sample size is large, you still may be able to use parametric techniques. We'll be doing a neat demonstration that conveys the role of large samples in salvaging data from the normal-distribution assumption.

To a large extent, the different non-parametric techniques represent analogues to parametric techniques. As one example, the non-parametric Mann-Whitney U test is analogous to the parametric t-test, when comparing data from two groups. I've even written a song for the occasion...

Mann-Whitney U (Video of Dr. Reifman performing it; thanks to Selen for filming)
Lyrics by Alan Reifman
(May be sung to the tune of “Suzie Q.,” Hawkins/Lewis, covered by John Fogerty)
Mann-Whitney U,
When your groups are two,
If your scaling’s suspect, and your cases are few,
Mann-Whitney U,

The cases are laid out,
Converted to rank scores,
You then add these up, done within each group,
Mann-Whitney U,

(Instrumental)

There is a formula,
That uses the summed ranks,
A distribution’s what you, compare the answer to,
Mann-Whitney U

2007-10-30T14:12:00.000-07:00

While grading the correlation assignments, I came across an interesting finding in one of the students' papers (we use the "GSS93 subset" practice data set in SPSS, and each student can select his or her own variables for analysis).

Among the variables selected by this one student were number of children and frequency of sex during the last year. At the bivariate, zero-order level, these two variables were correlated at r = -.102, p < .001 (n = 1,327).

The student then conducted a partial correlation, focusing on the same two variables, but this time controlling for age (the student used the four-category age variable, although a continuous age variable is also available). This partial correlation turned out to be r = .101, p < .001.

Having graded several dozen papers from this assignment over the years, my impression was that, at least among the variables chosen by my students from this data set, partialling out variables generally had little impact on the magnitude of correlation between the two focal variables. Granted, neither of the correlations in the present example are all that huge, but the changing of the correlation's sign from negative (zero-order) to positive (first-order partial), with each of the respective correlations significantly different from zero, was noteworthy in my mind.

Also of interest was that age had both a fairly substantial positive correlation with number of children (r = .437) and a comparably powerful negative correlation with frequency of sex (r = -.410).

To probe the difference between the zero-order and partial correlations between number of children and frequency of sex, I went back to the (PowerPoint) drawing board, and created a scatter plot, color coding for age (also, in scatter plots created in SPSS, a dot only appears to indicate the presence of at least one case at a given spot on the graph, not the number of cases, so I attempted to remedy that, too).

My plot is shown below (you can click to enlarge it). I added trend lines after studying the SPSS scatter plots to see where the lines would go. As can be seen, the full-sample trend is indeed of a negative correlation, whereas all the age-specific trends are positive. We'll discuss this further in class.

2007-10-11T11:13:00.000-07:00

Here's a more elaborate diagram (click to enlarge) of what I started sketching on the board at yesterday's class. It shows that, with smaller sample sizes, larger (absolute) values of r (i.e., further away from zero) are needed to attain statistical significance, than is the case with larger samples. In other words, with smaller samples, it takes a stronger correlation (in a positive or negative direction) to reject the null hypothesis of no true correlation in the full population (rho = 0) and rule out (to the degree of certainty indicated by the p level) that the correlation in your sample (r) has arisen purely from chance.

As you can see, statisticians sometimes talk about sample sizes in terms of degrees of freedom (df). We'll discuss df more thoroughly later in the course in connection with other statistical techniques. For now, though, suffice it to say that for ordinary correlations, df and sample size (N) are very similar, with df = N - 2 (i.e., the sample size, minus the number of variables in the correlation).

For a partial correlation that controls (holds constant) one variable beyond the two main variables being correlated (a first-order partial), df = N - 3; for one that controls for two variables beyond the two main ones (a second-order partial), df = N - 4, etc.

This web document also has some useful information.

2007-10-08T20:33:00.000-07:00

Last year's summary on correlational analysis is pretty thorough, so we'll largely use it. I just want to add a few things for this year:

First, here's a quote from the book Think of a Number (by Malcolm E. Lines, 1990) that puts the technique of partial correlation in some perspective:

It is probably the statistician's most difficult task of all to assure himself or herself that no unrecognized confounding factor lies hidden in the sampling groups which are being tested (p. 37).

Also, here are some correlation-related songs, for which I've written lyrics...

Fitting the Line (from one of my methods pages)

Restriction in the Range
Lyrics by Alan Reifman
(May be sung to the tune of “Laughter in the Rain,” Sedaka/Cody)

Why do you get such a small correlation,
With variables you think should be related?
Seems you’re not studying the full human spectrum,
Just looking at part of bivariate space,
All kinds of thoughts start to race, through your mind…

Ooh, there’s restriction in the range,
Dampening the slope of the best-fit line,
Ooh, I can correct r for this,
Put a better rho estimate in its place

Partial Correlation
Lyrics by Alan Reifman
(May be sung to the tune of “Crystal Blue Persuasion,” James/Vale/Gray)

Studying variables, with potential confounds,
Don’t want a paper, where confusion abounds,
There’s a technique, now, for consideration,
What you need to use, is partial correlation,

The connection of interest, is between A and B,
Each may be related, to the third factor, C,
There’s a formula out there, analysts would approve,
The influence of C, now, it will remove,

(Instrumental build-up)

Partial correlation,
For a strict-er determination,
Partial correlation,
A simple, calculation,

You’re purging C’s variance, from A and from B,
Thus from C’s linkage, the others are free,
The new A and B, now, you test for correlation,
The r, yes-sirree, AB-dot-C…

Partial correlation…

(Instrumental)

Partial correlation,
For a strict-er determination,
Partial correlation,
A simple, calculation,

Partial correlation ….
(Fade out)

[This website is excellent on partial correlation.]

Correlation (about correlation and causality, from one of my methods pages)

2007-09-26T12:00:00.000-07:00

We'll next be moving on to standardized (or z) scores and how they relate to the normal/bell curve and percentiles. Last year's notes on this topic, which also include some alternative norming schemes to the z-score, will be useful. This year, however, I wanted to add some information on one of the key aspects of z scores.

If we wanted to compare the performances of two or more individuals on some task, the ideal way, of course, would be to administer the same measures, under the same conditions, to everyone, and see who scores highest. Sometimes, however, it's clear that the people you're trying to compare have not been assessed under identical conditions.

As one example, a university may have a large, amphitheatre-type lecture class of 400 students, with each student also attending a TA-led discussion section of 25 students. There are eight TA's, each of whom leads two sections. Overall course grades may be based 80% on uniform in-class exams taken at the same time by all the students, and 20% on section performance (mini-paper assignments and spoken participation). The kicker is that, for the 20% of the grade that comes from the sections, different students have different TA's, who can differ in the toughness or easiness of their grading.

Another example, which comes from an actual published study, involves comparing the home-run prowess of sluggers from different eras. As those of you who are big baseball fans will know, Babe Ruth held the single-season home-run record for many years, with the 60 he hit in 1927. Roger Maris then came along with 61 in 1961, and that's where things stood for another few decades. Within the last decade, we then saw Mark McGwire hit 70 in 1998 and Barry Bonds belt 73 in 2001.

Given the many differences between the 1920s and now, is Bonds's 2001 season really the most impressive? The initial decades of the 1900s were known as the Dead-ball Era, due to the rarity of home runs. In contrast, the last several years have been dubbed "The Live-ball Era," "The Goofy-ball Era," "The Juiced-ball (or Juiced-player) Era," "The Steroid Era," etc. It's not just steroids that are suspected of inflating the home-run totals of contemporary batters; smaller stadiums, more emphasis on weightlifting, and league expansion (which necessitates greater use of inexperienced pitchers) have also been suggested as contributing factors.

A few years ago, a student named Kyle Bang (a great name for analyzing home runs) published an article in SABR's Baseball Research Journal, applying z-scores to the problem. Wrote Bang:

The z-score is best considered a measure of domination, since it only determines how well a hitter performed with respect to his contemporaries within the same season (p. 58).

In other words, for any given season, the home-run leader's total could be compared to the mean throughout baseball for the same season, with the difference divided by the standard deviation for the same season. A player with a high season-specific z score thus would have done well relative to all the other players that same season. (Bang actually used homer per at-bat as each player's input, so that, for example, a player's home-run performance would not be penalized if the player missed games due to injury.)

Under this method, Ruth has had the six best individual seasons in Major League Baseball history. The overall No. 1 season was Ruth's in 1920, where his z-score was 7.97 (i.e., Ruth's homer output in 1920 minus the major-league mean in 1920, and dividing the difference by the 1920 SD, equalled 7.97). Ruth's 1927 season, where his absolute number of homers established the record of 60, produced a z-score of 5.57, good for fourth on the list.

Bonds's 2001 season of 73 homers produced a z-score of 5.14, good for seventh on the list, and McGwire's 1998 season of 70 homers produced a z of 4.69, for eighth on the list.

Again, the point is that Ruth exceeded his contemporaries to a greater degree than Bonds and McGwire did theirs. The aforementioned factors that may have been inflating home-run totals in the 1990s and 2000s (steroids, small ballparks, etc.) would thus have helped Bonds and McGwire's contemporaries also hit lots of homers, thus raising the yearly means and weakening Bonds and McGwire's season-specific z-scores (although their z's were still pretty far out on the normal distribution).

As with any method, the z-score approach has its limitations. Among them, notes Bang, is that just by chance, some eras may have a lot of great hitters coming up at the same time, which raises the mean and weakens the top players' z-scores.

How would this logic apply to the example of the discussion sections of a large lecture class? Each TA could convert his or her students' orignal grades for section performance to z-scores, relative to that TA's grading mean and SD. If a particular TA were an easy grader, that TA's mean would be high and thus the top students would get their z-adjusted grade knocked down a bit. Conversely, a hard-grading TA would have a lower mean for his or her students, thus allowing them to get their grades bumped up a bit in the z-conversion.

Also, because z-scores have a mean of 0 and an SD of 1, and the section component would be counting 20% of students' overall course grades, the z-converted scores would have to be renormed. To get the students' section grades to top out at (roughly) 20, perhaps they could be converted to a system with a mean of 16 and an SD of 2.

The example of the discussion section grades is based on a true story. While in graduate school at the University of Michigan, I was a TA for a huge lecture class, and I suggested a z-based renorming of students' section grades. The professor turned the idea down, citing the increased complexity and grading time that would be involved.

I really believe the z-score approach gives you a lot of "bang" for your buck, but not everyone may agree.

2007-08-28T14:34:00.000-07:00

Welcome to HDFS 5349 Quantitative Methods I. This is my fifth straight year teaching this class in the Fall semester and the second year with a class blog.

One of the things I used the blog for last year was to create and post PowerPoint graphics that explained various statistical phenomena and how to interpret your computer (SPSS) output for some of the techniques. To a large extent, last year's graphics will play an important role again this year. I will, however, continue to post new entries this semester, as needed.

At Wednesday's opening class, we'll handle some administrative stuff such as taking roll. I'll also discuss some introductory issues, such as what statistics can tell us and what they cannot, and how our course (and other social science statistics courses) differ from the kinds of courses you would find in the math/statistics department.

I'll end this blog entry with a passage from a book I read earlier this year, Coincidences, Chaos, and All That Math Jazz, by Edward B. Burger and Michael Starbird.

Statistics can help us understand the world. It is a powerful and effective tool for placing economic, social welfare, sports, and health issues into perspective. It molds data into digestible morsels and shows us a measured way to look at situations that have either random or unknown features. But we must use common sense when applying statistics or other tools that draw on our experience of the world to shape data into meaningful conclusions (p. 60).

2007-03-14T22:20:00.000-07:00

I was talking with one of our graduate students, Jackie Wiersma, the other day about the availability of summer statistics courses throughout the nation, and the conversation inspired me to search for as many such programs as I could uncover and compile them here. Virtually all of these summer programs teach advanced statistics and methodology courses, which may seem out of place here on an Intro Stats blog, but this is as good a place as any for me to post this information.

The list below should definitely be considered a "work in progress." I will continue adding links to programs as I find out about them. Information can be e-mailed to me via my faculty webpage (link in the upper right).

These courses are typically available to graduate students and Ph.D. holders, although some have limited enrollment.

To any graduate students around the world who may see this information, if you're interested in taking a summer stats course at an institution other than your own, it would probably be a good idea for you to check with people at your home university to verify that credits from an outside institution will be able to count in your degree plan.

American Psychological Association Advanced Training Institutes. Short workshops (1-5 days), held at various locations in the U.S.; some deadlines have already passed (LINK).

Courses include: GIS, non-linear methods, web research, SEM-longitudinal, NICHD Early Child Care study.

NEW, ADDED 6/12/07
Claremont Graduate University (Evaluation and Applied Research Methods). Offers a wide array of one-day courses during the latter part of August (August 17-23 & 30, 2007), covering methodology, program evaluation, and statistics (LINK).

NEW, ADDED 7/19/07
Johns Hopkins University (Center for Prevention and Early Intervention). According to a message posted by Bengt Muthen to the APA Division 5 listserve: "Three one-day Mplus short courses will be held on August 20-22, 2007 at the Johns Hopkins University in Baltimore, MD. The three days cover (1) growth modeling, (2) multilevel regression, mediation analysis, factor analysis, and structural equation modeling, (3) multilevel latent class, latent transition and growth mixture modeling. The courses will be taught by Bengt and Linda Muthen" (LINK).

Northwestern University (Institute for Policy Research). Workshops on Quasi-Experimental Design and Analysis in Education; five-day workshops at various points during summer (one deadline has already passed, other sessions are still open); some expenses covered (LINK).

Notre Dame (Quantitative Psychology Program). Workshop on "Statistical Methods for Modeling Human Dynamics: An Interdisciplinary Dialogue," May 28-31 (LINK).

NEW, ADDED 5/6/07
Oregon State (Summer Institute on Research Methodology, College of Health and Human Sciences). A pair of two-day workshops are offered: July 9-10, introductory course on multilevel modeling; and July 11-12, advanced course on latent growth curve modeling (both using Mplus) (LINK).

Penn State (Summer Institute on Longitudinal Methods: Mixed Models for Longitudinal Continuous and Dichotomous Data and Practical Tools for Causal Inference). Application process is closed, but information may help you plan for future years. (LINK).

Statistical Horizons (Paul Allison). Two- and five-day workshops, mostly in Philadelphia; open to public, but also appears to have corporate aspect, with website alluding to capability of presenting courses at firms and to consulting services (LINK).

Courses include: survival analysis, categorical data analysis, and missing data.

Texas A&M University (Educational Psych). Five-day courses throughout most of June; registration by early-bird deadline of March 31 will save you some money (LINK).

Courses include: SEM, HLM, growth modeling, mixed methods, factor analysis, multivariate, and others.

University at Buffalo (Sociology). Five-day workshops from May 14-18 & 21-25 (LINK).

Courses include: SEM, HLM.

University of Connecticut (DATIC). Five-day workshops starting in late May and encompassing most of June (LINK).

Courses include: Dyadic analysis, SEM, HLM (two courses).

University of Kansas (Quantitative Psychology) "Stats Camp." Five-day courses, from June 4-8 & 11-15 (LINK).

Courses include: SEM, SEM-longitudinal, item response theory, and multilevel.

University of Maryland (Joint Program in Survey Methodology). Methodologically oriented courses (e.g., questionnaire design, data collection, sampling), as well as some on statistics/data analysis; primarily one- and four-week courses in June and July (LINK, then click on "Summer Mini Courses" in left-hand column).

University of Massachusetts (Center for Research on Families). Three- and five-day workshops in June and July, on topics relating to HLM, longitudinal trajectories, and dyadic data (LINK).

University of Michigan (ICPSR). Traditional four-week courses (June 25-July 20 & July 23-August 17), plus a seemingly growing number of one-week workshops (most of June & July, plus a little bit of August); some of the short workshops are held outside of Ann Arbor, Michigan (too bad -- just kidding), including in Bloomington, IN, Amherst, MA, Chapel Hill, NC, and New Haven, CT (LINK).

Courses include: Typical graduate-level techniques (e.g., regression, multivariate, SEM, HLM, longitudinal), plus specialized topics (e.g., long-term care, childcare/early ed, crime statistics).

University of Michigan (Public Health/Epidemiology). One- and three-week courses, from July 8-27; a blend of methodological/statistical topics (e.g., clinical trials, evaluation research, biostatistics) and public health-specific topics (e.g., disease, injuries, genetics) (LINK).

University of Michigan (Survey Research Center). Has more of a methodological focus than ICPSR (e.g., experimental and quasi-experimental design, questionnaire design, focus groups), but plenty of data-analysis courses are offered, as well; mostly four-week courses (June 4-29 & July 2-27), but also some one-week workshops (LINK).

Virginia Commonwealth University (CARMA program, School of Business). Three-day courses in May, as well as short courses and webcasts at other times of the year; discounts for individuals from organizations that are part of CARMA Webcast Consortium. (LINK).

Courses include: SEM, multilevel analysis, longitudinal, regression interactions, meta-analysis, Internet research, social networks, and grounded theory.

Yale University (Institution for Social & Policy Studies). July 11-13, workshops on "Designing, Conducting, and Analyzing Field Experiments" (LINK).

2007-03-08T00:07:00.000-08:00

Today I'm giving a guest lecture on survival analysis (also known as event history analysis) in my colleague Dr. Du Feng's graduate class on developmental (longitudinal) data analysis. Survival analysis is not a topic for introductory statistics, but this blog seemed to be as good a place as any for putting these web links to supplement my presentation.

Survival analysis is appropriate when a researcher has a dichotomous outcome variable that can be monitored at regular time intervals to see if each participant has switched from one status to another (e.g., in medical research, from alive to dead).

Here are a couple of documents I found on the web. First is a brief one from University College London that presents both a survival curve and a hazard curve, concepts that we shall discuss during class. Second is a more elaborate document from the University of Minnesota that discusses additional issues, including "censored" data.

The focus of my talk will be a class project from when I taught graduate research methods in the spring of 1998. The study led to a poster paper at the 1999 American Psychological Association conference (copies available upon request). We took advantage of the fact that People magazine, which comes out weekly and is archived in the Texas Tech library, lists celebrity marriages and divorces (and other developments) in a "Passages" section (here's an example, from after the study was completed).

We were looking at the survival of celebrity marriages until divorce. What makes the study a little unusual is that two events had to occur for a couple to provide complete data: the couple would have to get married, then get divorced. Quoting from our paper:

All issues of People between January 1, 1990 and June 30, 1997 were examined, for a total of 392 weeks. All marriages and divorces during this period were recorded, but only those couples whose marriages took place during the study period were used. To facilitate the survival analysis, for each couple the week number of the marriage and of the divorce (if any) were recorded...

Regarding the week numbers noted above, the issue of People dated January 1, 1990 represented week number 1, the January 8, 1990 issue represented week number 2, and so forth, up through the June 30, 1997 issue, which represented week number 392.

The particular type of survival analysis we conducted is called Cox Regression. Like other kinds of regression techniques, Cox Regression tests the relationship of predictor variables (covariates) to an outcome, in this case the hazard curve.

I hope you find this information useful and that everyone "survives" the lecture. For a nice, though somewhat dated, overview of the technique, I would recommend the following article:

Luke, D.A. (1993). Charting the process of change: A primer on survival analysis. American Journal of Community Psychology, 21, 203-246.

2006-11-26T19:07:00.000-08:00

We've previously discussed how we use analyses of sample statistics to make inferences about population parameters. If our sample-based analysis is not statistically significant, we say that the null hypothesis (Ho) of no correlation in the population, no difference between group means in the population, etc., is maintained. However, if the sample-based analysis reveals a significant correlation, mean difference, etc., we reject Ho and infer that some non-zero correlation or difference exists in the population.

"Non-zero" is not very specific, and we can do better! We cannot know an exact true population parameter (e.g., the population correlation rho) unless we survey everyone in the population, as done in the U.S. Census.

What we can do with our obtained sample statistic, however, is produce an estimate of the true population parameter, in the form of a confidence interval (CI). A 95% CI is most commonly used, corresponding to p < .05 significance. What we end up with is a range, where we can say that, if the full population were surveyed, we would be 95% confident the true population parameter would be inside that range. (The definition is actually a little more technical, as shown in this document, but the previous statement is a good approximation.)

The following chart presents visual depictions of 95% confidence intervals, based upon correlations reported in this article on children's physical activity. (On some computers, the image below may stall before its bottom portion appears; if you click on the image to enlarge it, you should get the full picture, which features a continuum of correlations from -.30 to .30 at the bottom.)

An important thing to notice from the chart is that there's a direct translation from a confidence interval around a sample correlation to its statistical significance. If a CI does not include zero (i.e., is entirely in positive "territory" or entirely in negative "territory"), then the result is significantly different from zero, and we can reject the null hypothesis of zero correlation in the population. On the other hand, if the CI does include zero (i.e., straddles positive and negative territory), then the result cannot be significantly different from zero, and Ho is maintained.

There are online calculators for obtaining CI's around different types of sample statistics (e.g., around a correlation r, around a mean, around a proportion). For the above example, of course, I used a calculator that could compute CI's around sample correlations (see the StatPages compendium in the links section on the right).

A well-known example of a confidence interval around a proportion is the margin of error in a political poll (here and here).

The dichotomous nature of null hypothesis significance testing (NHST) -- the idea that a result either is or is not significantly different from zero -- makes it less informative than the CI approach, where you get an estimated range within which the true population value is likely to fall. Therefore, many researchers have called for the abolition of NHST in favor of CI's (for an example, click here).

Recall, however, that even if only CI's are presented, an interpretation in terms of statistical significance can still be made. Thus, it seems, we can have our cake and eat it too! Of that you can be confident.

2006-11-17T09:01:00.000-08:00

Although we have not formally discussed the issue of statistical power, the general idea has come up many times. In the SPSS examples we've worked through, we sometimes have observed what look like small relationships or differences, but which have turned out to be statistically significant due to a large sample size. In other words, large sample sizes give you statistical power.

In research, our aim is to detect a statistically significant relationship (i.e., reject the null hypothesis) when the results warrant such. Therefore, my personal definition of statistical power boils down to just four words:

Detecting something that's there.

I also like to think in terms of a biologist attempting to detect some specimen with a microscope. Two things can aid in such detection: A stronger microscope (e.g., more powerful lenses, advanced technology) or a visually more apparent (e.g., clearer, darker, brighter) specimen.

The analogy to our research is that greater statistical power (i.e., increasing the sample size) is like increasing the strength of the microscope. Further, a stronger relationship in the data (e.g., a .70 correlation as opposed to .30, or a 10-point difference between two means as opposed to a 2.5 point difference) is like a more visually apparent specimen. Strength of the results is also somewhat under the control of the researchers, who can take steps such as using the most reliable and valid measures possible, avoiding range restriction when designing a correlational study, etc.

The calculation of statistical power can be informed by the following example. Suppose an investigator is trying to detect the presence or absence of something. For example, during the Olympics all athletes (or at least the medalists) will be tested for the presence or absence of banned substances in their bodies. Any given athlete either will or will not actually have a banned substance in his or her body (“true state”). The Olympic official, relying upon the test results, will render a judgment as to whether the athlete has or has not tested positive for banned substances (the decision). All the possible combinations of true states and human decisions can be modeled as followed (format based loosely on Hays, W.L., 1981, Statistics, 3rd ed.):

In other words, power is the probability that you’re not going to miss the fact the athlete truly has drugs in his/her system.

[Notes: The term “alpha” for a scale’s reliability is something completely separate and different from the present context. Also, the terms “Type I” and “Type II” error are sometimes used to refer, respectively, to false alarms and misses; I personally boycott the terms “Type I” and “Type II” because I feel they are very arbitrary, whereas you can reason out what a false alarm or miss is.]

For the kind of research you’ll probably be doing...

“The power of a test is the ability of a statistical test with a specified number of cases to detect a significant relationship.”

(Original source document for above quote no longer available online.)

Thus, you’re concerned with the presence or absence of a significant relationship between your variables, rather than the presence or absence of drugs in an athlete’s body.

This document makes two important points:

*There seems to be a consensus that the desired level of statistical power is at least .80 (i.e., if a signal is truly present, we should have at least an 80% probability of detecting it; note that random sampling error can introduce "noise" into our observations).

*Before you initiate a study, you can calculate the necessary sample size for a given level of power, or, if you're doing secondary analyses with an existing dataset, for example, you can calculate the power for the existing sample size. As the linked document notes, such calculations require you to input a number of study properties.

One that you'll be asked for is the expected "effect size" (i.e., magnitude of relationship). Here's where Cohen's classification of small, medium, and large relationships comes in handy (recall that we discussed this in the unit on correlation). I suggest being cautious, and assuming you'll discover only a small relationship in your upcoming study. For comparing two means, remember that the t-test is used only for determining statistical significance (i.e., seeing where your result falls on the t distribution). Thus, for a two-group comparison of means, you have to use something called "Cohen's d" for seeing what a small, medium, and large difference between means would be.

Power-related calculations can be done using online calculators via the Stat Pages compendium, shown in the links section on the right. There are power calculators specific to each type of statistical technique (e.g., power for correlational analysis, power for t-tests).

More power to you!

2006-11-08T11:41:00.000-08:00

I just got a colorful idea (to say the least) for how to illustrate the null hypothesis of a chi-square test. In the SPSS example we've been using, the principle behind Ho is that one group's (e.g., male) distribution into the different characteristics (marital statuses) will be equal to the other group's (female) distribution. The pie charts below serve as an illustration:

Again, the null hypothesis is that the male pie (and all its wedge sizes) will equal the female pie (and all its wedge sizes). A significant overall chi-square test tells us, of course, to reject Ho, which is to say, reject the notion of identical male and female pies.

A significant overall chi-square test can derive from any one category (wedge) or more being discrepant across males and females. That's where the standardized residuals for the cells can potentially be informative.

2006-11-06T23:00:00.000-08:00

We'll now be learning how to perform chi-square tests in SPSS. Most of the elements will be straightforward (observed and expected frequencies, the overall chi-square, degrees of freedom, and significance). There are a few aspects of the output that may be a little confusing, however, so I've made another handy-dandy guide to aid interpretation (below).

Perhaps the most confusing aspect of a chi-square table is how to report on percentages of respondents. The important thing to remember is that, in a statement of the form "the percentage of people in group A have characteristic B," the order of A and B is not interchangeable.

Here's an example. I would estimate that about 80% of the players in the National Basketball Association (NBA) are from the U.S. (players such as the Dallas Mavericks' Dirk Nowitzki and the Houston Rockets' Yao Ming are part of the growing international presence).

However, we would never claim the reverse -- that 80% of the people in the U.S. are NBA basketball players!

We thus have to be careful about phrasing the results of a chi-square analysis. A good practice is to request only ROW percentages from SPSS for the cells in the table. If you follow this practice, then you can always phrase your results in the following form. For any given cell, you can say something like:

"Among [category represented by the row], ___% [shown in cell] were [characteristic represented by the column]."

This format corresponds to the SPSS output in the following manner:

Update, November 13, 2007: Television host Keith Olbermann of MSNBC's "Countdown" awarded himself third place in his nightly "Worst Person in the World" competition. Olbermann's offense? He committed a statistical reversal error of the type described above. Quoting from the transcript of the show:

The bronze to me. We inverted a statistic last night. The study based on stats from the Veterans Affairs and Census Bureau indicating the heart breaking percentage of homeless veterans. I said one in every four veterans is homeless. In fact, one of every four homeless is a veteran. That makes the number smaller. It is, quote, only, unquote, 194,000; 1,500 of them, according to the V.A., veterans of Afghanistan and Iraq, already on the streets. I apologize for the statistical mistake.

Here are some other tips:

1. The statistical significance of a chi-square analysis pertains to the table as a whole. You're saying that the overall chi-square (which is the sum of the cell-specific chi-squares) exceeds the critical value for a given degrees of freedom and significance level.

2. To see if one or more cells are making particularly large contributions to the overall significance of the chi-square, you can have SPSS provide the unstandardized (regular) and standardized residuals for each cell. The regular residual is just the difference between the observed and expected frequency counts for a given cell. The standardized residual then puts the residual in z-score form, so that any standardized residual that's 1.96 or greater (in absolute value) could be said to be a major contributor to the overall chi-square. Standardized residuals may not be that informative, however, as with large sample sizes, many cells may have standardized residuals greater than 1.96. That makes it hard to pinpoint any one or two cells where the "action" is. For further information, see the links section to the right, under Chi-Square.

3. If your overall chi-square for an analysis is significant, you should describe your findings in a way that emphasizes contrast. For example: "Among women, 39% followed the election campaigns closely, whereas only 28% of men did."

2006-10-24T11:44:00.000-07:00

NOTE: I no longer advocate the approach to interpreting SPSS output for the Independent-Samples t-test that is depicted in the graphic below. I now support the approach in this entry (AR, 10/22/09).

Our next statistical technique is the t-test, for comparing two means and seeing if the difference is significant. Most of what we need is on my "Basic Statistics" lecture page for my research methods class (see links section on the right).

Previous years' experience suggests, however, that the SPSS printout for the independent-samples t-test is confusing to some students, so I have created a pictorial explanation (below). You can click on the picture to enlarge it and then, when it opens, an enlarger icon should appear to make it even bigger.

2006-10-15T11:51:00.000-07:00

We've now completed the unit on correlation and the significance testing thereof. I try my best to make my lectures as identical as possible for the two course sections, but inevitably, certain concepts and terms will receive greater coverage in one section than the other. Accordingly, I thought I'd summarize a few points on this blog, so everyone will have equal access to the information.

First, in order to have a statistically significant correlation, the correlation (r) itself should be appreciably different from zero, either above zero (a positive correlation) or below zero (a negative correlation).

Also, in order for the correlation to be significant, the significance (or probability) level displayed for a given correlation in your SPSS output must be very small (p < .05, or if the probability is even smaller, you can use one of the other conventional cut-off points, p < .01 or p < .001). Any time the probability p is larger than .05, the correlation is nonsignificant (in my opinion, if you get a correlation with a p level of .06 or .07, it's OK to note in your report that the correlation narrowly missed being significant under conventional standards).

Suppose you find that the correlation between two variables is r = .30, p < .01. This is telling us that, if the null hypothesis (Ho) is true -- that is, there truly is no correlation in the population from which the sample was drawn (rho = 0, where rho looks like a curvy capital P) -- then it would be extremely unlikely (p < .01) for a correlation of .30 to crop up purely by chance when the correlation throughout the society is truly zero.

[Here's a figure I've added in October 2007, to convey the idea of there truly being no correlation in a large population, but a correlation occurring in one's sample purely by random sampling error:

This web document is also helpful. The opposite problem, where the full population truly has a correlation, but you draw a sample that fails to show it, will be discussed later in the course.]

A significant correlation in our sample thus allows us to reject the null hypothesis and assert, based on an inference from our sample, that there is a correlation in the population. Again, note the inference from sample to population.

The essence of scientific hypothesis testing can thus be distilled to three steps:

1. State the null hypothesis (Ho) that there is no correlation between your two variables in the population (rho = 0). The investigator probably doesn't believe Ho (generally, we seek to uncover significant relationships), but Ho is part of the scientific protocol.

2. Obtain the sample correlation (r) between your two variables and the associated significance (p) level.

3. If the correlation is statistically significant (r is well above or well below zero, and p < .05), reject Ho. If the correlation is nonsignificant (r is close to zero and p is larger than .05), then the null hypothesis that there is no correlation between your two variables in the population must be maintained. We never accept the truth of the null hypothesis for certain; we just say it cannot be rejected.

I've stated above that, in order to be significant, a correlation (r) needed to be well above or well below zero. That's not always true, however. As we saw in some of our SPSS illustrations, with a very large sample size (n = 1,000 or more), a correlation does not necessarily need to be that far from zero to be significant. If a correlation appears to be small, yet is listed in the output as being significant (probably only due to the large sample size), you can say the correlation was "significant, though weak." A Wikipedia document on correlation (which I've just added to the links section on the right) displays guidelines developed by the late statistician Jacob Cohen for labeling correlations as "small, medium, and large."

Two concepts we will be taking up later in the course, statistical power and confidence intervals, will elaborate upon the issue of small correlations sometimes being significant and, conversely, relatively large correlations not being significant.

I also wanted to say a little more about partial correlations, which "hold constant" or "control for" one or more variables extraneous to your two primary variables. The idea I was trying to get across in class was that one or both of the primary variables may also be correlated with a third (or fourth, etc.) variable, which could obscure our understanding of the original two-variable relationship.

In the example we discussed in class (available in the links section), we speculated that the relationship between Body Mass Index (BMI) and success at In Vitro Fertilization (IVF) could be complicated by the possibility that BMI itself is positively correlated with age. A bivariate (or zero-order) correlation between BMI and IVF could be ambiguous because BMI might be carrying some influence of age.

(We also speculated about whether the age-IVF relationship might be complicated by the role of BMI, for which similar considerations as above would apply.)

The answer, of course, would be to obtain partial correlations (e.g., rBMI-IVF "dot" age). A third variable that may be correlated with one or both of the two primary variables in an analysis and thus could possibly complicate matters -- such as age in this example -- would be known as a "confound" or "confounder." "Con" means "with" or "together" in some languages, such as "arroz con pollo" (Spanish for "chicken and rice"). Confound thus means "found with" or "found together," such as age being "found with" BMI (i.e., we put on extra weight as we get older).

For anyone who is interested, this document by W.C. Burns probes the issues of confounds, third variables, and spurious relationships in greater depth.

Finally, I wanted to address a minor inconsistency in the SPSS outputs for bivariate (zero-order) and partial correlations. In the bivariate output, you get the correlation coefficient (r), significance (probability) level, and sample size (N) for each pair of variables. The partial correlation outputs likewise give you the partial correlation and significance level. However, instead of N, you get something closely related, called "degrees of freedom" (df). For correlational analyses, df simply equals sample size minus number of variables in the correlation (N - 2 for a bivariate correlation, N - 3 for a first-order partial correlation, N - 4 for a second-order partial, etc.).

2006-09-30T19:29:00.000-07:00

This coming week, we'll be learning about probability (King & Minium, 2003, Chapter 11). Probability is an important foundation for statistical analysis as, when you obtain a finding in your research (e.g., a higher percentage of couples show improvement after receiving a novel, experimental marital therapy than do control-group couples who received an older form of therapy), you need to know how likely it is this result came up purely by chance.

Although some of the topics in the chapter get fairly complex, the core idea to keep in mind regarding probability is how many possible ways can something turn out. A coin can come up heads or tails, thus the probability of a head is 1/2 (same for a tail).

A tool that has many useful applications in statistics and probability is the "n choose k" formulation (see the links section on the right for an online calculator). I've already alluded to this approach in talking about how last year's St. Louis University men's basketball team had a perfect alternation of wins and losses through its first 19 games (WLWLWLWLWLWLWLWLWLW). As discussed on my Hot Hand blog, I framed the question in the form of: How many possible different ways are there to distribute 10 items (wins) in 19 boxes (games)?

To simplify things, let's imagine that a sorority needs two of its members to oversee an event, but four members volunteer (Cammy, Pammy, Sammy, and Tammy). How many possible duos can be chosen from four members? This would be stated as "4 choose 2" (there would be a 4 on top of a 2 in parentheses, but this is not a fraction). Keeping in mind that n = 4 and k = 2, the following description from Ken Ross (p. 43) may be helpful:

...the denominator is the product of k numbers from k down to 1...

...the numerator is the product of numbers starting with n and going down until you also have k numbers in the numerator...

Mathematically, we would take:

4 X 3
_____

2 X 1

which equals 6. The possible duos are thus:

Cammy with Pammy
Cammy with Sammy
Cammy with Tammy
Pammy with Sammy
Pammy with Tammy
Sammy with Tammy

One of the most complicated areas in the chapter, if not the most complicated one, involves the binomial distribution (sections 11.4 and 11.5 of the chapter). I've created the following graphic to elaborate upon the big equation in Example 1 of p. 203 (note the presence of the "and/multiplication" and "or/addition" principles, first introduced on p. 199, and the "n choose k" principle).

Here's a little tip: Since you're adding up the probabilities of all possible occurrences, the big equation must sum to 1. If you get 1, that doesn't guarantee you did the calculation right, although you probably did. On the other hand, if you get an answer other than 1, you know you did it wrong!

Statisticians usually are interested in the probability of a result as or more extreme than that actually obtained (e.g., five or six correct responses in Example 1). The more specifically you define your outcome criteria (e.g., exactly five successes), the lower the probability of obtaining it by chance will be, so to avoid making our results look excessively rare, we would use the criterion of five or more successes.

There are also, I believe, two typographical errors in the chapter (footnote 1 on p. 202 and the very end of Example 1 on p. 203), which I will tell you about in class.

One of the last things covered in the chapter is the famous "Birthday Paradox." Upon first learning that a group size of only 23 people is necessary for the probability to be .50 that two of the people will have the same birthday, most observers find this very counterintuitive. The Wikipedia's page on the topic may help clarify the key points. One of the approaches taken on the Wikipedia page uses the "n choose k" principle. Another approach elaborates on the aforementioned "and/multiplication" principle that is briefly shown on p. 206 of our textbook. The probability of at least one pair of people having the same birthday is 1 minus the probability of no one having the same birthday. The latter can be thought of as the product of the following probabilities:

The first person definitely has his/her birthday on some day (1) X

The second person having it on one of the other 364 days of the year (364/365) X

The third person having his or hers on one of the remaining 363 days (363/365) X ...

2006-09-27T10:11:00.000-07:00

As you know, I like to find real-life examples of the concepts covered in class. Standard deviations, z scores, etc., rarely come up in the public discourse. One place they do come up, however, is in the criteria for membership in Mensa, also known as the "high IQ society" (click here and then scroll down a bit). Those of you who fly on American Eagle into and out of Lubbock, and read the American Way magazine on the plane, may have noticed that each issue includes some sample Mensa items, for your amusement and bemusement.

Update 9/27 late afternoon, after today's class: A couple things. First, to summarize how different distributions are normed to different means and standard deviations, here's a list (each distribution's properties can be reported in the form, M +/- SD):

..........................Mean.....Standard Deviation

Normal Distribution (z)...0................1.........

McCall T....................50.............10.........

IQ Tests..................100............15 or 16 (usually)

Here is the link for the "Flynn Effect" (rising population-level IQ's over the years), for those who are interested in it.

Also, Tuesday's class ended with a very elaborate normal-curve diagram I had drawn on the board. I think most students got it copied down OK, but I suggested that any students who had a camera, phone, or other devices with photographic capabilities might want to take a picture of the board, so I could post it on the class blog. Here it is...

I tell the students that, if you want to teach statistics some day, you're going to have to be able to draw good normal "bell-shaped" curves. The one above isn't too bad, but it's a bit flat at the top! (Note that "Column 3" in the drawing refers to Table A, beginning on p. 510 of the textbook, King & Minium, 2003.)

2006-09-19T20:52:00.000-07:00

As we finish up descriptive statistics (i.e., ways to describe distributions of data), I wanted to provide a conceptual scheme for those of you who want more structure to organize the information.

In my little example of how often people go to the movies, we saw how two distributions could have the same mean, but the distributions still had pretty different shapes (both were symmetrical, but one was more spread out than the other). The means don't tell the distributions apart, so we have to look at a second feature, the standard deviation, to tell them apart.

You can also have two distributions that have the same mean and same standard deviation, yet are still not identical. The following example from the financial world illustrates just such a case:

The blue and green distributions are clearly different. The distributions are not distinguished by their means (they're the same), nor are the distributions distinguished by their standard deviations (they're also the same). We have to look at a third feature of the distributions, their skewness, to tell them apart.

The logic that I hoped you've seized upon has to do with the mathematical concept of moments:

The mean is the first moment.

The standard deviation (or variance) is the second moment.

The skewness is the third moment.

The kurtosis (which your textbook briefly refers to on p. 44, 6th edition) is the fourth moment.

The textbook from when I took first-year graduate statistics in the psychology department at Michigan during the 1984-85 academic year (William L. Hays, Statistics, 3rd ed.) nicely summarizes moments, in my view:

Just as the mean describes the "location" of the distribution on the X axis, and the variance describes its dispersion, so do the higher moments reflect other features of the distribution. For example, the third moment about the mean is used in certain measures of degree of skewness... The fourth moment indicates the degree of "peakedness" or kurtosis of the distribution, and so on. These higher moments have relatively little use in elementary applications of statistics, but they are important for mathematical statisticians... The entire set of moments for a distribution will ordinarily determine the distribution exactly... (p. 167)

2006-09-18T11:21:00.000-07:00

The NBC television show Deal or No Deal is excellent for getting viewers to think in terms of probability. The new season debuts tonight, with additional new episodes being shown later in the week (schedule).

2006-09-13T09:41:00.000-07:00

We're going to move beyond simple histograms and focus more intensively on types of distributions. Figure 3.13 of your book (p. 44) nicely summarizes different possible shapes of distributions [updated to reflect the 6th edition of the textbook].

Much of statistics is based on the idea of a normal "bell-shaped" curve (chart "f" in the figure), where most participants have scores in the middle of the distribution (on whatever variable is being studied) and progressively fewer have scores as you move to the extremes (high and low) of the distribution. Variables such as self-identified political ideology in the U.S. and people's height have at least a roughly normal distribution. As discussed on p. 93 in a "Controversy" box in your book, however, data sets only rarely seem to follow the normal curve.

In recent years, however, there has been a lot of interest in "J-shaped" curves (chart "a"). Such curves, which are also known as "scale-free" or "power-law" distributions, represent the situation where most people have relatively low scores, a few people have moderately high scores, and a very few have extremely high scores. Examples of J-shaped curves include:

Number of sexual partners (see Figure 2 of linked document). The participants described in the linked study illustrate a J-shaped curve by themselves, but if you add in Gaetan Dugas, labeled as "Patient Zero" in the AIDS epidemic due to his reported 2,500 sex partners, you can see where the term "scale-free" comes from. With most people appearing to have 10 or fewer lifetime sexual partners, thus warranting increments of 1 on the X-axis, 2,500 is clearly "off the scale."

Wealth distributions. In the United States, "the top 1% of households (the upper class) owned 33.4% of all privately held wealth" (link). Thus, at the extreme high end of the X-axis, where the high monetary values would be located, the frequency of occurrence (Y-axis) would be very small (1%).

Alcohol consumption. See the reports here and here.

We will discuss the other types of distributions in the figure, as well.

2006-09-11T09:22:00.000-07:00

Now that we've gotten our foundations in sampling (which gives us the ability to generalize from a sample to the full population and thus engage in inferential statistics) and measurement (where knowing the types of variables in a study will help us determine what statistical technique to use), this week we will begin learning about how to describe the data in a sample.

Your textbook defines descriptive statistics as allowing the researcher to "organize and summarize observations" (King & Minium, p. 3). We will start out with percentiles, histograms, and distributions, and then cover statistics such as the mean and standard deviation.

As a cute, real-world example of "organizing and summarizing observations," a Canadian student named Andy Saunders created a blog during the Jeopardy winning streak of Ken Jennings (which stretched 74 games) to provide statistical data on everything anybody would ever want to know about the Jeopardy games played by Jennings.

Here's another story that previews some of the issues we will be addressing in the class. With the five-year anniversary of 9/11, a lot of media coverage has been devoted to the topic. A piece I saw on CNN asked whether the occurrence of disease in many people who worked at Ground Zero in the days after 9/11 and thus were exposed to potentially toxic dust particles is higher than would be expected by chance (or higher than in occupationally similar people who were not exposed to particles at Ground Zero). The show's transcript is available here, and you'll have to scroll pretty far down to see the story on 9/11 workers.

2006-08-28T09:10:00.000-07:00

Welcome to Quantitative Methods I (Intro Stats) and, for those of you who haven't been in school here in the past, welcome to Texas Tech!

I'll do my best to provide a lot of practical, real-world exercises in analyzing data, and I'll try to keep things fun. In this class, you'll learn what I mean by such phrases as "Paula Abdul significance" and "t for two," and how the statistical concept of "degrees of freedom" is nicely illustrated by one of the games on The Price is Right.

The main purposes of this blog are to compile web links of relevance to statistics (in the right-hand column) and enable me to post miscellaneous notes. Good luck this semester!