// Numbas version: exam_results_page_options
{"name": "Marlon's copy of Adaptive marking: Independent two sample t-test", "extensions": ["stats"], "custom_part_types": [], "resources": [], "navigation": {"allowregen": true, "showfrontpage": false, "preventleave": false, "typeendtoleave": false}, "question_groups": [{"pickingStrategy": "all-ordered", "questions": [{"functions": {"pstdev": {"definition": "sqrt(len(l)/(len(l)-1))*stdev(l)", "type": "number", "language": "jme", "parameters": [["l", "list"]]}}, "ungrouped_variables": [], "name": "Marlon's copy of Adaptive marking: Independent two sample t-test", "tags": ["average", "data analysis", "differences", "elementary statistics", "hypothesis testing", "mean", "standard deviation", "statistics", "stats", "t-test", "two sample t-test", "variance"], "preamble": {"css": "", "js": ""}, "advice": "<p>We test the following hypothesis,</p>\n<p style=\"text-align: center;\">$H_0:\\; \\mu_1=\\mu_2$ versus $H_1:\\; \\mu_1 \\neq \\mu_2$</p>\n<p>We find that the mean score of Group 1 is $\\overline{x}_1=\\var{mean1}$ with standard deviation $s_1=\\var{sd1}$ and the mean score of Group 2 is $\\overline{x}_2=\\var{mean2}$ with standard deviation $s_2=\\var{sd2}$.</p>\n<p>(All calculated to 3 decimal places.)</p>\n<p>Using the formula for the two-sample $t$-statistic as &nbsp;shown above with $n_1=n_2=10$:</p>\n<p>The estimate of the pooled variance is calculated to be:</p>\n<p>\\[s^2=\\frac{(n_1-1)s_1^2+(n_2-1)s_2^2}{n_1+n_2-2}= \\frac{\\var{n1-1}\\times \\var{sd1}^2+\\var{n2-1}\\times \\var{sd2}^2}{\\var{n1+n2-2}}=\\var{s^2}.\\]&nbsp;</p>\n<p>Hence $s = \\sqrt{\\var{s^2}}=\\var{s}$ to 3 decimal places.</p>\n<p>We find that the t-statistic has value:</p>\n<p>\\begin{align}<br/>T &amp;= \\frac{(\\overline{x}_1-\\overline{x}_2)-(\\mu_1-\\mu_2)}{s\\sqrt{\\frac{1}{n_1}+\\frac{1}{n_2}}} \\\\<br/>&amp;= \\frac{(\\var{mean1}-\\var{mean2})-(0)}{\\var{s}\\sqrt{\\frac{1}{\\var{n1}}+\\frac{1}{\\var{n2}}}} \\\\<br/>&amp;= \\var{t_statistic}<br/>\\end{align}</p>\n<p>Our test statistic is $|T|=\\var{abs(t_statistic)}$.</p>\n<p>Given that we have $n_1+n_2-2=18$ degrees of freedom, we look up this value on the T-distribution table for $t_{18}$</p>\n<p>\\[\\begin{array}{r|rrrrr}&amp;0.10&amp;0.05&amp;0.01&amp;0.001\\\\\\hline18&amp;1.734&amp;2.101&amp;2.878&amp;3.922\\end{array}\\]</p>\n<p>We see that the t-statistic {t_statistic_range} and the table tells us that the $p$ value {p_value_range}.</p>\n<p>Hence we conclude that we {reject} the null hypothesis. There is {evidence_strength} evidence of a difference between the average scores of the two groups.</p>", "rulesets": {"std": ["all", "fractionNumbers", "!collectNumbers", "!noLeadingMinus"]}, "parts": [{"stepsPenalty": 0, "prompt": "<p>Find the mean and standard deviations of the scores of the two groups.&nbsp;Round your answers to 3 decimal places.</p>\n<table>\n<thead>\n<tr><th></th><th>Mean</th><th>Standard deviation</th></tr>\n</thead>\n<tbody>\n<tr><th>Group 1</th>\n<td>[[0]]</td>\n<td>[[1]]</td>\n</tr>\n<tr><th>Group 2</th>\n<td>[[2]]</td>\n<td>[[3]]</td>\n</tr>\n</tbody>\n</table>\n<p>Now find the two sample <em>t</em>-test statistic $T$ using the values you have just calculated and enter it here: [[4]]</p>", "variableReplacements": [], "variableReplacementStrategy": "originalfirst", "gaps": [{"precisionType": "dp", "precisionMessage": "You have not given your answer to the correct precision.", "allowFractions": false, "variableReplacements": [], "precision": "3", "maxValue": "mean1", "variableReplacementStrategy": "originalfirst", "strictPrecision": false, "correctAnswerFraction": false, "showCorrectAnswer": true, "precisionPartialCredit": 0, "scripts": {}, "marks": 0.5, "minValue": "mean1", "type": "numberentry", "showPrecisionHint": false}, {"precisionType": "dp", "precisionMessage": "You have not given your answer to the correct precision.", "allowFractions": false, "variableReplacements": [], "precision": "3", "maxValue": "sd1", "variableReplacementStrategy": "originalfirst", "strictPrecision": false, "correctAnswerFraction": false, "showCorrectAnswer": true, "precisionPartialCredit": 0, "scripts": {}, "marks": 0.5, "minValue": "sd1", "type": "numberentry", "showPrecisionHint": false}, {"precisionType": "dp", "precisionMessage": "You have not given your answer to the correct precision.", "allowFractions": false, "variableReplacements": [], "precision": "3", "maxValue": "mean2", "variableReplacementStrategy": "originalfirst", "strictPrecision": false, "correctAnswerFraction": false, "showCorrectAnswer": true, "precisionPartialCredit": 0, "scripts": {}, "marks": 0.5, "minValue": "mean2", "type": "numberentry", "showPrecisionHint": false}, {"precisionType": "dp", "precisionMessage": "You have not given your answer to the correct precision.", "allowFractions": false, "variableReplacements": [], "precision": "3", "maxValue": "sd2", "variableReplacementStrategy": "originalfirst", "strictPrecision": false, "correctAnswerFraction": false, "showCorrectAnswer": true, "precisionPartialCredit": 0, "scripts": {}, "marks": 0.5, "minValue": "sd2", "type": "numberentry", "showPrecisionHint": false}, {"precisionType": "dp", "precisionMessage": "You have not given your answer to the correct precision.", "allowFractions": false, "variableReplacements": [{"variable": "mean1", "part": "p0g0", "must_go_first": false}, {"variable": "sd1", "part": "p0g1", "must_go_first": false}, {"variable": "mean2", "part": "p0g2", "must_go_first": false}, {"variable": "sd2", "part": "p0g3", "must_go_first": false}], "precision": "3", "maxValue": "t_statistic", "variableReplacementStrategy": "originalfirst", "strictPrecision": false, "correctAnswerFraction": false, "showCorrectAnswer": true, "precisionPartialCredit": 0, "scripts": {}, "marks": 1, "minValue": "t_statistic", "type": "numberentry", "showPrecisionHint": false}], "steps": [{"prompt": "<p>The two-sample <em>t</em>-statistic for two independent sets of data where one set has $n_1$ data points and the other set $n_2$ data points is calculated as follows:</p>\n<p>\\[T = \\frac{(\\overline{x}_1-\\overline{x}_2)-(\\mu_1-\\mu_2)}{s\\times\\sqrt{\\frac{1}{n_1}+\\frac{1}{n_2}}}\\;\\;\\;\\]</p>\n<p>where $\\overline{x}_1,\\;\\overline{x}_2$ are the sample means and&nbsp;</p>\n<p>\\[s^2=\\frac{(n_1-1)s_1^2+(n_2-1)s_2^2}{n_1+n_2-2}\\]</p>\n<p>where $s_1,\\;s_2$ are the sample standard deviations.</p>\n<p>Use the values you calculated to 3 decimal places in order to find $T$.</p>", "variableReplacements": [], "variableReplacementStrategy": "originalfirst", "showCorrectAnswer": true, "scripts": {}, "marks": 0, "type": "information"}], "scripts": {}, "marks": 0, "showCorrectAnswer": true, "type": "gapfill"}, {"displayColumns": "1", "prompt": "<p>Give the value $|T|$ of the <em>t</em>-statistic you have found, choose the range for the $p$ value by looking up the <em>t</em>&nbsp;tables:</p>", "matrix": "decision_marking_matrix", "shuffleChoices": false, "variableReplacements": [{"variable": "t_statistic", "part": "p0g4", "must_go_first": false}], "choices": ["<p>$p$ is less than $0.1\\%$</p>", "<p>$p$ lies between $0.1\\%$ and $1\\%$</p>", "<p>$p$ lies between  $1 \\%$ and $5\\%$</p>", "<p>$p$ lies between  $5 \\%$ and $10\\%$</p>", "<p>$p$ is greater than $10\\%$</p>"], "variableReplacementStrategy": "alwaysreplace", "displayType": "radiogroup", "maxMarks": 0, "scripts": {}, "marks": 0, "showCorrectAnswer": true, "type": "1_n_2", "minMarks": 0}, {"displayColumns": 0, "prompt": "<p>Given the $p$-value and the range you have found, what is the strength of evidence against the null hypothesis that there is no difference in the average times for the left and right hands?</p>", "matrix": "decision_marking_matrix", "shuffleChoices": false, "variableReplacements": [{"variable": "scenario", "part": "p1", "must_go_first": false}], "choices": ["<p>Very Strong Evidence</p>", "<p>Strong Evidence</p>", "<p>Evidence</p>", "<p>Weak Evidence</p>", "<p>No Evidence</p>"], "variableReplacementStrategy": "alwaysreplace", "displayType": "radiogroup", "maxMarks": 0, "scripts": {}, "marks": 0, "showCorrectAnswer": true, "type": "1_n_2", "minMarks": 0}, {"displayColumns": "1", "prompt": "<p>What do you&nbsp;decide&nbsp;based on the above analysis?</p>", "matrix": "decision_marking_matrix", "shuffleChoices": false, "variableReplacements": [{"variable": "scenario", "part": "p2", "must_go_first": false}], "choices": ["<p>We reject the null hypothesis at the $0.1\\%$ level</p>", "<p>We reject the null hypothesis at the $1\\%$ level.</p>", "<p>We reject the null hypothesis at the $5\\%$ level.</p>", "<p>We do not reject the null hypothesis but consider further investigation.</p>", "<p>We do not reject the null hypothesis.</p>"], "variableReplacementStrategy": "alwaysreplace", "displayType": "radiogroup", "maxMarks": 0, "scripts": {}, "marks": 0, "showCorrectAnswer": true, "type": "1_n_2", "minMarks": 0}], "statement": "<p>An educational psychologist claimed that the order in which questions were asked affected the&nbsp;student&rsquo;s ability to answer them correctly and hence their total score. In order to test this, $20$ students were randomly divided into two groups of $10$. The first group were given questions&nbsp;in increasing order of difficulty and the second group in decreasing order of difficulty. The&nbsp;ordered test scores obtained were:</p>\n<table border=\"1\">\n<tbody>\n<tr><th align=\"left\">Group 1</th>\n<td>{r1[0]}</td>\n<td>{r1[1]}</td>\n<td>{r1[2]}</td>\n<td>{r1[3]}</td>\n<td>{r1[4]}</td>\n<td>{r1[5]}</td>\n<td>{r1[6]}</td>\n<td>{r1[7]}</td>\n<td>{r1[8]}</td>\n<td>{r1[9]}</td>\n</tr>\n<tr><th align=\"left\">Group 2</th>\n<td>{r2[0]}</td>\n<td>{r2[1]}</td>\n<td>{r2[2]}</td>\n<td>{r2[3]}</td>\n<td>{r2[4]}</td>\n<td>{r2[5]}</td>\n<td>{r2[6]}</td>\n<td>{r2[7]}</td>\n<td>{r2[8]}</td>\n<td>{r2[9]}</td>\n</tr>\n</tbody>\n</table>\n<p>Carry out a two-sample <em>t</em>-test to decide&nbsp;if there is evidence of a difference in the average&nbsp;test scores for the two sets of students.</p>", "variable_groups": [{"variables": ["n1", "n2", "mu1", "sigma1", "mu2", "sigma2"], "name": "Setup"}, {"variables": ["r1", "r2"], "name": "Samples"}, {"variables": ["mean1", "sd1", "mean2", "sd2", "s", "t_statistic", "p_value"], "name": "Stats"}, {"variables": ["scenario", "decision_marking_matrix", "reject", "evidence_strength", "t_statistic_range", "p_value_range"], "name": "Advice messages"}, {"variables": ["t90", "t95", "t99", "t999"], "name": "Critical t-values"}], "variablesTest": {"maxRuns": 100, "condition": ""}, "variables": {"sd1": {"definition": "precround(pstdev(r1),3)", "templateType": "anything", "group": "Stats", "name": "sd1", "description": "<p>Sample standard deviation of sample 1</p>"}, "sd2": {"definition": "precround(pstdev(r2),3)", "templateType": "anything", "group": "Stats", "name": "sd2", "description": "<p>Sample standard deviation of sample 2</p>"}, "t_statistic": {"definition": "(mean1-mean2)*sqrt(n1*n2)/(s*sqrt(n1+n2))", "templateType": "anything", "group": "Stats", "name": "t_statistic", "description": ""}, "sigma1": {"definition": "random(8..10#0.2)", "templateType": "anything", "group": "Setup", "name": "sigma1", "description": "<p>Population standard deviation of sample 1</p>"}, "sigma2": {"definition": "random(8..10#0.2)", "templateType": "anything", "group": "Setup", "name": "sigma2", "description": "<p>Population standard deviation of sample 2</p>"}, "mean2": {"definition": "mean(r2)", "templateType": "anything", "group": "Stats", "name": "mean2", "description": "<p>Sample&nbsp;mean of sample 1</p>"}, "scenario": {"definition": "sum(map(abs(t_statistic)<x,x,[t90,t95,t99,t999]))", "templateType": "anything", "group": "Advice messages", "name": "scenario", "description": "<p>Which scenario are we in - how many critical values of the t distribution does t_statistic exceed?</p>"}, "reject": {"definition": "if(scenario<2,'do reject','do not reject')", "templateType": "anything", "group": "Advice messages", "name": "reject", "description": "<p>Do we reject the null hypothesis?</p>"}, "p_value_range": {"definition": "['is less than $0.001$','lies between $0.001$ and $0.01$','lies between $0.01$ and $0.05$','lies between $0.05$ and $0.10$','is greater than $0.10$'][scenario]", "templateType": "anything", "group": "Advice messages", "name": "p_value_range", "description": "<p>Describe where the p-value lies in relation to the critical values</p>"}, "evidence_strength": {"definition": "['very strong','strong','slight','no','no'][scenario]", "templateType": "anything", "group": "Advice messages", "name": "evidence_strength", "description": "<p>How much evidence is there against the null hypothesis?</p>"}, "t999": {"definition": "3.922", "templateType": "anything", "group": "Critical t-values", "name": "t999", "description": ""}, "t99": {"definition": "2.878", "templateType": "anything", "group": "Critical t-values", "name": "t99", "description": ""}, "decision_marking_matrix": {"definition": "[\n  [1,0,0,0,0],\n  [0,1,0,0,0],\n  [0,0,1,0,0],\n  [0,0,0,1,0],\n  [0,0,0,0,1]\n][scenario]", "templateType": "anything", "group": "Advice messages", "name": "decision_marking_matrix", "description": "<p>Marking matrix for the multiple choice questions</p>"}, "t95": {"definition": "2.101", "templateType": "anything", "group": "Critical t-values", "name": "t95", "description": ""}, "t90": {"definition": "1.734", "templateType": "anything", "group": "Critical t-values", "name": "t90", "description": ""}, "p_value": {"definition": "ttest(abs(t_statistic),19,2)", "templateType": "anything", "group": "Stats", "name": "p_value", "description": "<p>p-value&nbsp;corresponding to the t-statistic</p>"}, "mu1": {"definition": "random(55..75#0.5)", "templateType": "anything", "group": "Setup", "name": "mu1", "description": "<p>Population mean of sample 1 (we'll generate samples from different distributions to produce different outcomes)</p>"}, "r1": {"definition": "repeat(round(normalsample(mu1,sigma1)),n1)", "templateType": "anything", "group": "Samples", "name": "r1", "description": "<p>Sample 1</p>"}, "r2": {"definition": "repeat(round(normalsample(mu2,sigma2)),n2)", "templateType": "anything", "group": "Samples", "name": "r2", "description": "<p>Sample 2</p>"}, "mu2": {"definition": "random(65..75#0.5)", "templateType": "anything", "group": "Setup", "name": "mu2", "description": "<p>Population mean of sample 2</p>"}, "t_statistic_range": {"definition": "['is greater than $\\\\var{t999}$','lies between $\\\\var{t99}$ and $\\\\var{t999}$','lies between $\\\\var{t95}$ and $\\\\var{t99}$','lies between $\\\\var{t90}$ and $\\\\var{t95}$','is less than $\\\\var{t90}$'][scenario]", "templateType": "anything", "group": "Advice messages", "name": "t_statistic_range", "description": "<p>Describe where the t-statistic lies in relation to the critical values</p>"}, "s": {"definition": "precround(sqrt(((n1-1)*sd1^2+(n2-1)*sd2^2)/(n1+n2-2)),3)", "templateType": "anything", "group": "Stats", "name": "s", "description": "<p>Used in the&nbsp;formula for the t statistic</p>"}, "n1": {"definition": "10", "templateType": "anything", "group": "Setup", "name": "n1", "description": "<p>Size of sample 1</p>"}, "n2": {"definition": "10", "templateType": "anything", "group": "Setup", "name": "n2", "description": "<p>Size of sample 2</p>"}, "mean1": {"definition": "mean(r1)", "templateType": "anything", "group": "Stats", "name": "mean1", "description": "<p>Sample mean of sample 1</p>"}}, "metadata": {"notes": "<p>The student's values for the means and&nbsp;standard deviations are used to mark the t-statistic.</p>\n<p>The student's t-statistic is used to mark the p-value selection.</p>\n<p>The options for the multiple choice questions correspond to the <code>scenario</code> variable, so we can carry errors forward by replacing the value of that variable with the student's answers to each of the previous multiple choice parts. So, if the student incorrectly assesses&nbsp;the strength of the evidence based on their p-value, the question checks that they make a decision consistent with their assessment.</p>", "description": "<p>Two sample t-test to see if there is a difference between scores on questions between two groups when the questions are asked in a different order.</p>", "licence": "Creative Commons Attribution 4.0 International"}, "type": "question", "showQuestionGroupNames": false, "question_groups": [{"name": "", "pickingStrategy": "all-ordered", "pickQuestions": 0, "questions": []}], "contributors": [{"name": "Marlon Arcila", "profile_url": "https://numbas.mathcentre.ac.uk/accounts/profile/321/"}]}]}], "contributors": [{"name": "Marlon Arcila", "profile_url": "https://numbas.mathcentre.ac.uk/accounts/profile/321/"}]}