Can automated item generation be used to develop high quality MCQs that assess application of knowledge?

Research and Practice in Technology Enhanced Learning

Table 3 Frequency distributions for item quality metrics by item modality

		Strongly disagree	Disagree	Neutral	Agree	Strongly agree
The central idea is in the stem	AIG	1	4	4	14	181
The central idea is in the stem	Traditional	2	5	5	25	167
Directions in the stem are very clear	AIG	0	3	4	9	188
Directions in the stem are very clear	Traditional	0	7	4	22	171
No obvious cues or item flaws	AIG	9	9	1	12	173
No obvious cues or item flaws	Traditional	4	15	3	13	169
Length of choices about equal	AIG	4	10	2	19	169
Length of choices about equal	Traditional	3	10	3	28	160
All distractors are plausible	AIG	7	24	9	34	130
All distractors are plausible	Traditional	2	25	3	43	131
This is a high-quality item	AIG	7	33	22	50	92
This is a high-quality item	Traditional	8	38	20	56	82