Two versions of the independent 2-sample t-test
The difference lies in your assumption about the variances of the 2 underlying populations
In statistics, there are actually two versions of the independent 2-sample t-test, and you may be using the wrong one. Do you know which version you should be using for your next statistical analysis?
These 2 versions differ on their assumption about the variances of the 2 underlying populations:
One version assumes that the 2 populations have equal variances; this is called Student's t-test.
One version does NOT assume that the 2 populations have equal variances; this is called Welch's t-test.
Student's t-test is likely more well-known and perhaps more commonly used, but its assumption about equal variances is often false in practice. If you are not sure that this assumption holds for your data, then use Welch's t-test. If they turn out to be the same, then Welch's t-test will still be correct. This is a great advantage of Welch's t-test.
Daniel Lakens is an Associate Professor in the Human-Technology interaction group at Eindhoven University of Technology. As he advocated on his blog, we should always use Welch's t-test.
The only disadvantage about Welch's t-test is its formula for the number of degrees of freedom; it is much more complicated than that of Student's t-test. (This is likely why Student’s t-test is more well-known; it is easier to teach in an introductory class about statistics.) However, when performing t-tests using statistical programming languages like SAS, Python, or R, you don't need to worry about that. The built-in functions for Welch's t-test take care of this calculation.
In SAS, you can use PROC TTEST. Welch’s t-test is part of the default output for a 2-sample t-test; it has the label "Satterthwaite”.
In Python, the scipy.stats.ttest_ind() function can implement either version, depending on whether the "equal_var" parameter is True or False. By default, it assumes that the variances are equal (“equal_var=True”).
In R, a similar syntax applies to the t.test() function and its “var.equal” parameter. However, the default is the opposite: This function does NOT assume that the variances are equal (“var.equal = FALSE”).