Significance testing - Effective Base - Revision history

Mtabadmin at 00:21, 28 November 2013

2013-11-28T00:21:42Z

Mtabadmin at 00:19, 28 November 2013

2013-11-28T00:19:58Z

Mtabadmin at 00:18, 28 November 2013

2013-11-28T00:18:33Z

Mtabadmin at 00:17, 28 November 2013

2013-11-28T00:17:42Z

Mtabadmin at 19:20, 27 November 2013

2013-11-27T19:20:17Z

Mtabadmin at 19:19, 27 November 2013

2013-11-27T19:19:36Z

Mtabadmin at 19:10, 27 November 2013

2013-11-27T19:10:56Z

Mtabadmin at 19:08, 27 November 2013

2013-11-27T19:08:24Z

Mtabadmin: Created page with "When tstats are specified on weighted tables, Quantum calculates and uses the effective base in the calculation of the test statistics. The following explains why the effectiv..."

2013-11-27T18:44:31Z

Created page with "When tstats are specified on weighted tables, Quantum calculates and uses the effective base in the calculation of the test statistics. The following explains why the effectiv..."

New page

When tstats are specified on weighted tables, Quantum calculates and uses the effective base in the calculation of the test statistics. The following explains why the effective base is used and how it is calculated.

Consider the case in which the data is weighted by sex (male and female), to adjust it to reflect the population being studied, because 30% women and 70% men were sampled when the actual population proportions are 52% women and 48% men.

In this case, the weighting is actually inflating the answers for the women and deflating the answers for the men in order to match the population proportions. In other words, an answer from a woman will count as more than 1 on the tables, similarly, an answer from a man will count as less than 1, to be precise, every woman will count as 1.733 and every an as 0.686.

The effective base is used as a safeguard against making statistical conclusions from a sample that has been drastically adjusted (using weights) to match the population.

The effective base is calculated using the following formula,

effective base = (sum of weight factors) squared / sum of the squared weight factors.

If the data for a particular column is as follows

Total unweighted 40

Total weighted 40

and comes from 12 women and 28 men, then the effective base is

effective base = (12*1.733 + 28*0.686)**2 / (12*(1.733**2)+(28*(0.686)**2)

= 1600 / 49.2162 = 32.509

The effective base is a good criterion to judge how good the weighting is. If the weighting is inflating the answers from a particular group by a large factor, the effective base will end to be much smaller than the unweighted and weighted bases.

The closer the effective base is to the unweighted base, the better the weighting is.

@@ Line 9: / Line 9: @@
 effective base = (sum of weight factors) squared / sum of the squared weight factors.
-If the data for a particular column is as follows
+If the data for a particular column is as follows:
 Total unweighted 40
@@ Line 26: / Line 26: @@
 |}
-then the effective base is
+then the effective base is:
 effective base = (12*1.7333 + 28*0.6857)**2 / (12*(1.7333**2)+(28*(0.6857)**2)

@@ Line 3: / Line 3: @@
 Weighting actually inflates the "number of answers" per respondents, at times by a relatively large factor.  When performing significance testing, we ideally wish to utilize the weighted data to capture the intent of the weighting, however, as significance testing is impacted by the scale of the weighted values, weighted significance testing can be impacted adversely by weighting, in particular in cases in which weights represent large factors, say x10 or x100 or more.
-In these cases, the effective base calculation can remove the influence of the numerically large weighted values while capturing the intent of the weighting.  Basically, effective base is used as a safeguard against making statistical conclusions from a sample that has been drastically adjusted (using weights) to match the population.
+In these cases, the effective base calculation can remove the influence of the numerically large weighted values while capturing the intent of the weighting.  Basically, effective base is used as a safeguard against making statistical conclusions from a sample that has been drastically adjusted (using weights) to match target values like sales or population.
-The effective base is calculated using the following formula:
 effective base = (sum of weight factors) squared / sum of the squared weight factors.

@@ Line 22: / Line 22: @@
 Applying Effective Base:
 From mTAB's spreadsheet view, select menu items Data, then Significance Indicators.
 Click on the "Effective Base" checkbox from the resultant Significance Indicator dialog.

@@ Line 1: / Line 1: @@
 Survey data is often weighted, at times to balance sampling bias (e.g. obtain a market-distribution representation of gender and age) and at other times weighting to obtain a representation of sales, population, etc.
-Weighting actually inflates the "number of answers" per respondents, at times by a relatively large factor.  When performing significance testing, we ideally wish to utilize the weighted data to capture the intent of the weighting, however, as test testing is impacted by the scale of the weighted values, weighted significance testing can be impacted adversely by weighting, in particular in cases in which weights represent large factors, say x10 or x100 or more.
+Weighting actually inflates the "number of answers" per respondents, at times by a relatively large factor.  When performing significance testing, we ideally wish to utilize the weighted data to capture the intent of the weighting, however, as significance testing is impacted by the scale of the weighted values, weighted significance testing can be impacted adversely by weighting, in particular in cases in which weights represent large factors, say x10 or x100 or more.
 In these cases, the effective base calculation can remove the influence of the numerically large weighted values while capturing the intent of the weighting.  Basically, effective base is used as a safeguard against making statistical conclusions from a sample that has been drastically adjusted (using weights) to match the population.

← Older revision		Revision as of 19:10, 27 November 2013
Line 20:		Line 20:

	= 1600 / 49.2162 = 32.509		= 1600 / 49.2162 = 32.509
		+
		+	Applying Effective Base:
		+	From mTAB's spreadsheet view, select menu items Data, then Significance Indicators.
		+	Click on the "Effective Base" checkbox from the resultant Significance Indicator dialog.

@@ Line 14: / Line 14: @@
 Total weighted 40
 and comes from 12 women and 28 men, then the effective base is
-effective base = (12*1.733 + 28*0.686)**2 / (12*(1.733**2)+(28*(0.686)**2)
+effective base = (12*1.7333 + 28*0.6857)**2 / (12*(1.7333**2)+(28*(0.6857)**2)
 = 1600 / 49.2162 = 32.509