Table 2 Rationale for choice of quality metrics
Items included in Gierl & Lai paper (2013) | Rationale for including or excluding |
|---|---|
Every item should reflect specific content, as outlined in the test specifications | Excluded because all our items are mapped to objectives Also, unreasonable to expect participants to know all our objectives |
The question is based on important topics in the curriculum and is designed to measure key thinking and problem-solving skills | Excluded because it requires two judgments and we replaced with a question about problem-solving skills |
The question is carefully edited, formatted, and presented using correct grammar, punctuation, capitalization, and spelling | Excluded because this is a high-stakes exam, and all items are professionally edited |
The central idea is included in the stem, not the options | Included, but re-worded to respect the original language |
The stem of the question is worded positively and avoids negatives such as NOT and EXCEPT | Excluded as there are no negatively worded items in our active test bank |
Only one of the options is clearly correct | Excluded as this may be difficult for our panel to assess |
The correct option is not cued by item writing errors | Included, but re-worded to respect the original language |
All of the distractors are plausible | Included, but re-worded to respect the original language |