For Multiple Course Sections and Semesters 

1. Decide which aspects of a learning outcome to assess

To which aspects of a given Learning Outcome does your course contribute? In other words, which specific outcome-related concepts, skills, and procedures do your students need to know in your course? For example, a statistics course may end up selecting a set of 25 statistical concepts and procedures that embody "the quantitative skills necessary to evaluate and solve business problems." (GLO 3.1)

2. Decide on desired mastery levels

Specify at what level students should demonstrate mastery of the expected outcome. Consider the following levels:

1. Knowledge, i.e. recall of facts

Example:       Which of the following is NOT a type of schizophrenia?

(a)  Hebephrenic

(b)  Catatonic

(c)   Paranoid

(d)  Autistic

2. Application, i.e. using a concept in the context of a real task

Example:       Calculate the arithmetic mean for the following set of numbers:

                                       10 - 8 - 7 - 5 - 3 - 2 - 4 - 1

(a)  8

(b)  7

(c)   6

(d)  5

3. Analysis, i.e. breaking a scenario or a phenomenon into different aspects and thereby arriving at an appropriate conclusion

                        Example:       Which of the following conditions would most likely contribute to a state of high inflation?

(a)  High growth rate and low unemployment

(b)  Higher taxes and high unemployment

(c)   Low growth rate and low unemployment

(d)  Lower taxes and high unemployment

3. Guidelines for writing test items

Use the following guidelines to write effective test items:

1. Write items as simply as possible
2. Test for important ideas, information and skills—not trivial detail
3. Avoid using direct quotations or verbatim statements from your textbook
4. Avoid questions where the answer depends on the correct answer to a previous question
5. Ask questions about single ideas; avoid double-barreled questions
6. Avoid negative wording. If you do use negative words, underline and/or capitalize
7. Avoid use of "always," "never," and "all." Those are almost always indicators of an incorrect choice
8. When drafting your multiple choice options, write the correct response first; then grammatically match the other responses
9. Be consistent in the number of options used. Four alternatives are recommended because it is difficult to provide more than three plausible distracters
10. Provide only one correct or best answer. Avoid "all of the above," "none of the above," etc.
11. Options should be similar in length. Too often the longest choice is the most correct
12. Be sure that correct options are randomly distributed.

4. Create tests of similar difficulty levels

To create a good test item bank, calculate each item's difficulty level after students have responded to them in a test. This allows you to avoid giving tests in different semesters that have different difficulty levels. The difficulty index is simply the proportion of students who get an item correct. If for a given item, twenty students out of a class of forty-five choose the correct option, the difficulty index is:  20/45 = 0.44

a.      Difficulty indices run from 0 to 1.00. The larger the index, the easier the item.

b.      Test items should have difficulties in the vicinity of .50, because at this level the best discrimination values are obtained.

c.       Therefore, create the bulk of your test items with a difficulty index in the range of .30 to .70.

d.      However, don't not rule out items outside this range. High-difficulty items allow inference of superior understanding.  Low-difficulty items identify the level of deficiency that the poorest students exhibit.

e.      Creating a good test-item bank for reliably assessing program outcomes over multiple years requires (1) multiple items that address the same concept/skill/procedure, and (2) items of equal difficulty level for those concepts/skills/procedures.

 More detail can be found at:

 L.C. Jacobs & C.I. Chase. (1992). Developing and using tests effectively. San Francisco: Jossey-Bass.
J.C. Ory & K.E. Ryan. (1993). Tips for improving testing and grading. Newbury Park, CA: Sage

Back to Assessment Home