// Numbas version: exam_results_page_options
{"name": "Two sample t-test: question order and scores", "extensions": ["stats"], "custom_part_types": [], "resources": [], "navigation": {"allowregen": true, "showfrontpage": false, "preventleave": false, "typeendtoleave": false}, "question_groups": [{"pickingStrategy": "all-ordered", "questions": [{"name": "Two sample t-test: question order and scores", "tags": [], "metadata": {"description": "<p>This question assesses learners' ability to conduct a two sample t-test.&nbsp;</p>", "licence": "Creative Commons Attribution-ShareAlike 4.0 International"}, "statement": "<p>A study was conducted to investigate whether the order in which questions are asked affects students' ability to answer correctly.</p>\n<p>$20$ students were randomly divided into two groups of $10$. The first group were given questions&nbsp;in increasing order of difficulty and the second group in decreasing order of difficulty. The test scores obtained were:</p>\n<table border=\"1\">\n<tbody>\n<tr>\n<th align=\"left\">Group 1</th>\n<td>{r1[0]}</td>\n<td>{r1[1]}</td>\n<td>{r1[2]}</td>\n<td>{r1[3]}</td>\n<td>{r1[4]}</td>\n<td>{r1[5]}</td>\n<td>{r1[6]}</td>\n<td>{r1[7]}</td>\n<td>{r1[8]}</td>\n<td>{r1[9]}</td>\n</tr>\n<tr>\n<th align=\"left\">Group 2</th>\n<td>{r2[0]}</td>\n<td>{r2[1]}</td>\n<td>{r2[2]}</td>\n<td>{r2[3]}</td>\n<td>{r2[4]}</td>\n<td>{r2[5]}</td>\n<td>{r2[6]}</td>\n<td>{r2[7]}</td>\n<td>{r2[8]}</td>\n<td>{r2[9]}</td>\n</tr>\n</tbody>\n</table>\n<p>Let's use a two-sample <em>t</em>-test to test the hypothesis that there is a difference in the average test scores in relation to question order.</p>", "advice": "<p>Review two sample t-tests, e.g., section 7.3 in&nbsp;<a href=\"https://www.openintro.org/book/os/\" target=\"_blank\" rel=\"noopener noreferrer\">OpenIntro Statistics</a>&nbsp;(4th ed).</p>\n<p>We test the following hypothesis,</p>\n<p style=\"text-align: center;\">$H_0:\\; \\mu_1=\\mu_2$ versus $H_1:\\; \\mu_1 \\neq \\mu_2$</p>\n<p>We find that the mean score of Group 1 is $\\overline{x}_1=\\var{mean1}$ with standard deviation $s_1=\\var{sd1}$ and the mean score of Group 2 is $\\overline{x}_2=\\var{mean2}$ with standard deviation $s_2=\\var{sd2}$.</p>\n<p>Using the formula for the two-sample $t$-statistic as &nbsp;shown above with $n_1=n_2=10$:</p>\n<p>The standard error is:</p>\n<p>\\[SE=\\sqrt{\\frac{s_1^2}{n_1}+\\frac{s_2^2}{n_2}}=\\var{s}.\\]&nbsp;</p>\n<p>The T statistic is:</p>\n<p>\\begin{align}<br/>T &amp;= \\frac{(\\overline{x}_1-\\overline{x}_2)-(\\mu_1-\\mu_2)}{SE} \\\\<br/>&amp;= \\frac{(\\var{mean1}-\\var{mean2})-(0)}{\\var{s}} \\\\<br/>&amp;= \\var{t_statistic}<br/>\\end{align}</p>\n<p>Our test statistic is $|T|=\\var{abs(t_statistic)}$.</p>\n<p>We can assume to have $n_1-1=9$ degrees of freedom.</p>\n<p>Now use the T.DIST.2T Excel function to obtain the p value.</p>\n<p>We see that the $p$ value {p_value_range}.</p>\n<p>Hence we conclude that we {reject} the null hypothesis; there is {evidence_strength} evidence of a difference between the average scores of the two groups.</p>", "rulesets": {"std": ["all", "fractionNumbers", "!collectNumbers", "!noLeadingMinus"]}, "extensions": ["stats"], "builtin_constants": {"e": true, "pi,\u03c0": true, "i": true}, "constants": [], "variables": {"scenario": {"name": "scenario", "group": "Samples", "definition": "sum(map(if(abs(t_statistic)<x, 1, 0),x,[t90,t95,t99,t999]))", "description": "<p>Which scenario are we in - how many critical values of the t distribution does t_statistic exceed?</p>", "templateType": "anything", "can_override": false}, "t99": {"name": "t99", "group": "Stats", "definition": "3.250", "description": "<p>Critical t value for 1% tail probability, two tails, nDOFs = 9.</p>", "templateType": "anything", "can_override": false}, "mean2": {"name": "mean2", "group": "Setup", "definition": "mean(r2)", "description": "<p>Sample&nbsp;mean of sample 1</p>", "templateType": "anything", "can_override": false}, "mu1": {"name": "mu1", "group": "Setup", "definition": "random(55..75#0.5)", "description": "<p>Population mean of sample 1 (we'll generate samples from different distributions to produce different outcomes)</p>", "templateType": "anything", "can_override": false}, "evidence_strength": {"name": "evidence_strength", "group": "Samples", "definition": "['very strong','strong','slight','no','no'][scenario]", "description": "<p>How much evidence is there against the null hypothesis?</p>", "templateType": "anything", "can_override": false}, "sigma1": {"name": "sigma1", "group": "Setup", "definition": "random(8..10#0.2)", "description": "<p>Population standard deviation of sample 1</p>", "templateType": "anything", "can_override": false}, "t999": {"name": "t999", "group": "Stats", "definition": "4.781", "description": "<p>Critical t value for 0.1% tail probability, two tails, nDOFs = 9.</p>", "templateType": "anything", "can_override": false}, "p_value_range": {"name": "p_value_range", "group": "Samples", "definition": "['is less than $0.001$','lies between $0.001$ and $0.01$','lies between $0.01$ and $0.05$','lies between $0.05$ and $0.10$','is greater than $0.10$'][scenario]", "description": "<p>Describe where the p-value lies in relation to the critical values</p>", "templateType": "anything", "can_override": false}, "tol": {"name": "tol", "group": "Setup", "definition": "0.05", "description": "", "templateType": "anything", "can_override": false}, "n2": {"name": "n2", "group": "Setup", "definition": "10", "description": "<p>Size of sample 2</p>", "templateType": "anything", "can_override": false}, "sd2": {"name": "sd2", "group": "Setup", "definition": "siground(pstdev(r2),3)", "description": "<p>Sample standard deviation of sample 2</p>", "templateType": "anything", "can_override": false}, "sigma2": {"name": "sigma2", "group": "Setup", "definition": "random(8..10#0.2)", "description": "<p>Population standard deviation of sample 2</p>", "templateType": "anything", "can_override": false}, "t90": {"name": "t90", "group": "Stats", "definition": "1.833", "description": "<p>Critical t value for 10% tail probability, two tails, nDOFs = 9.</p>", "templateType": "anything", "can_override": false}, "reject": {"name": "reject", "group": "Samples", "definition": "if(scenario<2,'do reject','do not reject')", "description": "<p>Do we reject the null hypothesis?</p>", "templateType": "anything", "can_override": false}, "t_statistic_range": {"name": "t_statistic_range", "group": "Samples", "definition": "['is greater than $\\\\var{t999}$','lies between $\\\\var{t99}$ and $\\\\var{t999}$','lies between $\\\\var{t95}$ and $\\\\var{t99}$','lies between $\\\\var{t90}$ and $\\\\var{t95}$','is less than $\\\\var{t90}$'][scenario]", "description": "<p>Describe where the t-statistic lies in relation to the critical values</p>", "templateType": "anything", "can_override": false}, "r2": {"name": "r2", "group": "Samples", "definition": "repeat(round(normalsample(mu2,sigma2)),n2)", "description": "<p>Sample 2</p>", "templateType": "anything", "can_override": false}, "r1": {"name": "r1", "group": "Samples", "definition": "repeat(round(normalsample(mu1,sigma1)),n1)", "description": "<p>Sample 1</p>", "templateType": "anything", "can_override": false}, "decision_marking_matrix": {"name": "decision_marking_matrix", "group": "Samples", "definition": "[\n  [1,0,0,0,0],\n  [0,1,0,0,0],\n  [0,0,1,0,0],\n  [0,0,0,1,0],\n  [0,0,0,0,1]\n][scenario]", "description": "<p>Marking matrix for the multiple choice questions</p>", "templateType": "anything", "can_override": false}, "t_statistic": {"name": "t_statistic", "group": "Setup", "definition": "siground(abs((mean1-mean2)/s),3)", "description": "", "templateType": "anything", "can_override": false}, "p_value": {"name": "p_value", "group": "Setup", "definition": "siground(ttest(abs(t_statistic),n1,2),3)", "description": "<p>p-value&nbsp;corresponding to the t-statistic</p>", "templateType": "anything", "can_override": false}, "mu2": {"name": "mu2", "group": "Setup", "definition": "random(65..75#0.5)", "description": "<p>Population mean of sample 2</p>", "templateType": "anything", "can_override": false}, "n1": {"name": "n1", "group": "Setup", "definition": "10", "description": "<p>Size of sample 1</p>", "templateType": "anything", "can_override": false}, "sd1": {"name": "sd1", "group": "Setup", "definition": "siground(pstdev(r1),3)", "description": "<p>Sample standard deviation of sample 1</p>", "templateType": "anything", "can_override": false}, "mean1": {"name": "mean1", "group": "Setup", "definition": "mean(r1)", "description": "<p>Sample mean of sample 1</p>", "templateType": "anything", "can_override": false}, "t95": {"name": "t95", "group": "Stats", "definition": "2.262", "description": "<p>Critical t value for 5% tail probability, two tails, nDOFs = 9.</p>", "templateType": "anything", "can_override": false}, "s": {"name": "s", "group": "Setup", "definition": "precround(sqrt(sd1^2/n1+sd2^2/n2),3)", "description": "<p>Standard error - used in the&nbsp;formula for the t statistic.</p>", "templateType": "anything", "can_override": false}}, "variablesTest": {"condition": "", "maxRuns": 100}, "ungrouped_variables": [], "variable_groups": [{"name": "Setup", "variables": ["n1", "n2", "mu1", "sigma1", "mu2", "sigma2", "tol", "mean1", "sd1", "mean2", "sd2", "s", "t_statistic", "p_value"]}, {"name": "Samples", "variables": ["r1", "r2", "scenario", "decision_marking_matrix", "reject", "evidence_strength", "t_statistic_range", "p_value_range"]}, {"name": "Stats", "variables": ["t90", "t95", "t99", "t999"]}], "functions": {"pstdev": {"parameters": [["l", "list"]], "type": "number", "language": "jme", "definition": "sqrt(len(l)/(len(l)-1))*stdev(l)"}}, "preamble": {"js": "", "css": ""}, "parts": [{"type": "gapfill", "useCustomName": true, "customName": "Critical t-values", "marks": 0, "scripts": {}, "customMarkingAlgorithm": "", "extendBaseMarkingAlgorithm": true, "unitTests": [], "showCorrectAnswer": true, "showFeedbackIcon": true, "variableReplacements": [], "variableReplacementStrategy": "originalfirst", "nextParts": [], "suggestGoingBack": false, "adaptiveMarkingPenalty": 0, "exploreObjective": null, "prompt": "<p>Find the mean and standard deviations of the scores of the two groups.</p>\n<table>\n<thead>\n<tr>\n<th></th>\n<th>Mean</th>\n<th>Standard deviation</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<th>Group 1</th>\n<td>[[0]]</td>\n<td>[[1]]</td>\n</tr>\n<tr>\n<th>Group 2</th>\n<td>[[2]]</td>\n<td>[[3]]</td>\n</tr>\n</tbody>\n</table>\n<p>Now find the absolute value of the two sample <em>t</em>-test statistic:</p>\n<p>$|T|=$&nbsp;[[4]]</p>", "gaps": [{"type": "numberentry", "useCustomName": false, "customName": "", "marks": 0.5, "scripts": {}, "customMarkingAlgorithm": "", "extendBaseMarkingAlgorithm": true, "unitTests": [], "showCorrectAnswer": true, "showFeedbackIcon": true, "variableReplacements": [], "variableReplacementStrategy": "originalfirst", "nextParts": [], "suggestGoingBack": false, "adaptiveMarkingPenalty": 0, "exploreObjective": null, "minValue": "mean1*(1-tol)", "maxValue": "mean1*(1+tol)", "correctAnswerFraction": false, "allowFractions": false, "mustBeReduced": false, "mustBeReducedPC": 0, "displayAnswer": "", "showFractionHint": true, "notationStyles": ["plain", "en", "si-en"], "correctAnswerStyle": "plain"}, {"type": "numberentry", "useCustomName": false, "customName": "", "marks": 0.5, "scripts": {}, "customMarkingAlgorithm": "", "extendBaseMarkingAlgorithm": true, "unitTests": [], "showCorrectAnswer": true, "showFeedbackIcon": true, "variableReplacements": [], "variableReplacementStrategy": "originalfirst", "nextParts": [], "suggestGoingBack": false, "adaptiveMarkingPenalty": 0, "exploreObjective": null, "minValue": "sd1*(1-tol)", "maxValue": "sd1*(1+tol)", "correctAnswerFraction": false, "allowFractions": false, "mustBeReduced": false, "mustBeReducedPC": 0, "displayAnswer": "", "showFractionHint": true, "notationStyles": ["plain", "en", "si-en"], "correctAnswerStyle": "plain"}, {"type": "numberentry", "useCustomName": false, "customName": "", "marks": 0.5, "scripts": {}, "customMarkingAlgorithm": "", "extendBaseMarkingAlgorithm": true, "unitTests": [], "showCorrectAnswer": true, "showFeedbackIcon": true, "variableReplacements": [], "variableReplacementStrategy": "originalfirst", "nextParts": [], "suggestGoingBack": false, "adaptiveMarkingPenalty": 0, "exploreObjective": null, "minValue": "mean2*(1-tol)", "maxValue": "mean2*(1+tol)", "correctAnswerFraction": false, "allowFractions": false, "mustBeReduced": false, "mustBeReducedPC": 0, "displayAnswer": "", "showFractionHint": true, "notationStyles": ["plain", "en", "si-en"], "correctAnswerStyle": "plain"}, {"type": "numberentry", "useCustomName": false, "customName": "", "marks": 0.5, "scripts": {}, "customMarkingAlgorithm": "", "extendBaseMarkingAlgorithm": true, "unitTests": [], "showCorrectAnswer": true, "showFeedbackIcon": true, "variableReplacements": [], "variableReplacementStrategy": "originalfirst", "nextParts": [], "suggestGoingBack": false, "adaptiveMarkingPenalty": 0, "exploreObjective": null, "minValue": "sd2*(1-tol)", "maxValue": "sd2*(1+tol)", "correctAnswerFraction": false, "allowFractions": false, "mustBeReduced": false, "mustBeReducedPC": 0, "displayAnswer": "", "showFractionHint": true, "notationStyles": ["plain", "en", "si-en"], "correctAnswerStyle": "plain"}, {"type": "numberentry", "useCustomName": false, "customName": "", "marks": "5", "scripts": {}, "customMarkingAlgorithm": "", "extendBaseMarkingAlgorithm": true, "unitTests": [], "showCorrectAnswer": true, "showFeedbackIcon": true, "variableReplacements": [{"variable": "mean1", "part": "p0g0", "must_go_first": false}, {"variable": "sd1", "part": "p0g1", "must_go_first": false}, {"variable": "mean2", "part": "p0g2", "must_go_first": false}, {"variable": "sd2", "part": "p0g3", "must_go_first": false}], "variableReplacementStrategy": "originalfirst", "nextParts": [], "suggestGoingBack": false, "adaptiveMarkingPenalty": 0, "exploreObjective": null, "minValue": "t_statistic*(1-tol)", "maxValue": "t_statistic*(1+tol)", "correctAnswerFraction": false, "allowFractions": false, "mustBeReduced": false, "mustBeReducedPC": 0, "displayAnswer": "", "showFractionHint": true, "notationStyles": ["plain", "en", "si-en"], "correctAnswerStyle": "plain"}], "sortAnswers": false}, {"type": "1_n_2", "useCustomName": false, "customName": "", "marks": 0, "scripts": {}, "customMarkingAlgorithm": "", "extendBaseMarkingAlgorithm": true, "unitTests": [], "showCorrectAnswer": true, "showFeedbackIcon": true, "variableReplacements": [{"variable": "t_statistic", "part": "p0g4", "must_go_first": false}], "variableReplacementStrategy": "alwaysreplace", "nextParts": [], "suggestGoingBack": false, "adaptiveMarkingPenalty": "0", "exploreObjective": null, "prompt": "<p>Calculate the $p$ value and select the corresponding range:</p>", "minMarks": 0, "maxMarks": 0, "shuffleChoices": false, "displayType": "radiogroup", "displayColumns": "1", "showCellAnswerState": true, "choices": ["<p>$p$ is less than $0.1\\%$</p>", "<p>$p$ lies between $0.1\\%$ and $1\\%$</p>", "<p>$p$ lies between  $1 \\%$ and $5\\%$</p>", "<p>$p$ lies between  $5 \\%$ and $10\\%$</p>", "<p>$p$ is greater than $10\\%$</p>"], "matrix": "decision_marking_matrix"}, {"type": "1_n_2", "useCustomName": false, "customName": "", "marks": 0, "scripts": {}, "customMarkingAlgorithm": "", "extendBaseMarkingAlgorithm": true, "unitTests": [], "showCorrectAnswer": true, "showFeedbackIcon": true, "variableReplacements": [{"variable": "scenario", "part": "p1", "must_go_first": false}], "variableReplacementStrategy": "alwaysreplace", "nextParts": [], "suggestGoingBack": false, "adaptiveMarkingPenalty": "0", "exploreObjective": null, "prompt": "<p>Given the $p$-value and the range you have found, what is the strength of evidence against the null hypothesis that there is no difference in the average scores for the two groups?</p>", "minMarks": 0, "maxMarks": 0, "shuffleChoices": false, "displayType": "radiogroup", "displayColumns": 0, "showCellAnswerState": true, "choices": ["<p>Very Strong Evidence</p>", "<p>Strong Evidence</p>", "<p>Evidence</p>", "<p>Weak Evidence</p>", "<p>No Evidence</p>"], "matrix": "decision_marking_matrix"}, {"type": "1_n_2", "useCustomName": false, "customName": "", "marks": 0, "scripts": {}, "customMarkingAlgorithm": "", "extendBaseMarkingAlgorithm": true, "unitTests": [], "showCorrectAnswer": true, "showFeedbackIcon": true, "variableReplacements": [{"variable": "scenario", "part": "p2", "must_go_first": false}], "variableReplacementStrategy": "alwaysreplace", "nextParts": [], "suggestGoingBack": false, "adaptiveMarkingPenalty": "0", "exploreObjective": null, "prompt": "<p>What do you&nbsp;decide&nbsp;based on the above analysis?</p>", "minMarks": 0, "maxMarks": 0, "shuffleChoices": false, "displayType": "radiogroup", "displayColumns": "1", "showCellAnswerState": true, "choices": ["<p>We reject the null hypothesis at the $0.1\\%$ level</p>", "<p>We reject the null hypothesis at the $1\\%$ level.</p>", "<p>We reject the null hypothesis at the $5\\%$ level.</p>", "<p>We do not reject the null hypothesis but consider further investigation.</p>", "<p>We do not reject the null hypothesis.</p>"], "matrix": "decision_marking_matrix"}], "partsMode": "all", "maxMarks": 0, "objectives": [], "penalties": [], "objectiveVisibility": "always", "penaltyVisibility": "always", "contributors": [{"name": "Christian Lawson-Perfect", "profile_url": "https://numbas.mathcentre.ac.uk/accounts/profile/7/"}, {"name": "Mario Orsi", "profile_url": "https://numbas.mathcentre.ac.uk/accounts/profile/427/"}]}]}], "contributors": [{"name": "Christian Lawson-Perfect", "profile_url": "https://numbas.mathcentre.ac.uk/accounts/profile/7/"}, {"name": "Mario Orsi", "profile_url": "https://numbas.mathcentre.ac.uk/accounts/profile/427/"}]}