Background on Q3

Test specialists need to assign item parameters to each item and an examinee parameter to each examinee so that they can probabilistically predict the response of any examinee to any item, even if similar examinees have never taken similar items before. For such a purpose, Item Response Theory (IRT) has been developed and has popularly used for item analysis, test construction, and test equating. Most commonly used 3-parameter logistic IRT model takes the mathematical form:

where $a_i$ is the item discrimination parameter, is the item difficulty parameter, $c_i$ is the guessing parameter, and is the probability that an examinee with ability level $\theta$ answers item i correctly. When an IRT model is used to analyze the data, ignoring the violation of these assumptions, the estimated values of item parameters and the examinee parameter may not be accurate. Moreover, the major usage of the most standardized achievement test is selection, prediction and placement. When the use of a test results can affect the life chances or educational opportunities of examinees, The analysis of test data should be done carefully. Therefore, Several methods for assessing local dependence in the test data have been developed. Yen [49] proposed that Q3 index is useful to detect local dependence. Q3 is the correlation of a pair of items with the trait estimate partialled out. Yen illustrated that the residual is calculated as

where $u_{ik}$ is the score of the examinee on the $i^{th}$ item. can be estimated by equation 6.6. Then Q3 is the correlation of deviation scores across all examinees and can be expressed as

$$\mathbf{Q3_{ij} = r_{d_id_j}}$$ (8)

for item i and j, and it provides a pair-wise index of item dependence. If local independence is held between any pair of items, expected value of Q3 is supposed to be close to , where is the number of items on the test [50]. In [12], Q3 index has successfully used to provide an evidence of local dependence.