TEST DEVELOPMENT PROCESS


Steps in Test Development
 
 

  1. Conceptualization


    2. Have to think about: What do I want to measure? Is it worth measuring?
     

  2. Construction


    3.             Scaling
                            Types of scales
                                                a. Summative Rating Scales e.g., Likert
                                                b. Unidimensional vs Multidimensional Scales

                            Scaling methods

                                                A.  Yield ordinal data

                                                       a. Ratings
                                                        b. Paired comparisons
                                                        c. Sorting procedures

                                                                Comparative (paired comparison / ranking)
                                                                Categorical e.g. Guttman

                                                B.    Yield interval data

                                                        a. Thurstone scale


            Writing Items
                        Content
                        Format
                                    Selected response (multiple choice, matching, true-false)
                                    Constructed response (completion, short answer, essay)

                        Item pool size
                        Scoring procedures (cumulative, class, ipsative)
 
 

  1. Tryout


    2. Pilot sample (size and composition)
     
     

  2. Item Analysis (What makes a "Good" item)


    3. A. Quantitative Methods

            Item-Difficulty Index
            Item- Validity Index
            Item-Reliability Index
                    Factor analysis & inter-item consistency
            Item-Discrimination Index
                    Analysis of item alternatives (effectiveness of distractors)
            Item Characteristic Curves (graphs item difficulty and discrimination)
                    Latent-trait models (skip)
            Other things to consider
                    Guessing
                    Item fairness
                    Speeded tests


        B. Qualitative Methods
 

            The "think aloud" test administration
            Expert panels
 
 

  1. Revision


    2.     Cross - Validation








    | PSYC 200 | | PSYC 307 | | PSYC 308 | | WEB BOARD |
    | EMAIL ME | | GERONTOLOGY | | PSYC DEPT | | NEIU LIBRARY |