Computerized Adaptive Test-related Functions – `catsim.cat`

Functions used specifically during the application/simulation of computerized adaptive tests.

Compute the test bias, an evaluation criterion for computerized adaptive test methodologies [Chang2001].

The value is computed as:

\[Bias = \frac{\sum_{i=1}^{N} (\hat{\theta}_i - \theta_{i})}{N}\]

where \(\hat{\theta}_i\) is examinee \(i\) estimated ability and \(\theta_i\) is examinee \(i\) actual ability. A positive bias indicates systematic overestimation, while a negative bias indicates systematic underestimation.

Parameters:

actualnpt.ArrayLike: Array-like (list, tuple, or numpy array) containing the true ability values (float type).
predictednpt.ArrayLike: Array-like (list, tuple, or numpy array) containing the estimated ability values (float type).

Returns:

float: The bias between the predicted values and actual values.

Raises:

ValueError: If actual and predicted vectors are not the same size.

catsim.cat.dodd(theta: float, item_bank: ItemBank, correct: bool) → float[source]

Estimate \(\hat{\theta}\) when the response vector is composed entirely of 1s or 0s.

Method proposed by [Dod90]. This heuristic prevents the maximum likelihood estimator from returning infinity when all responses are correct or negative infinity when all responses are incorrect.

\[\begin{split}\hat{\theta}_{t+1} = \left\lbrace \begin{array}{ll} \hat{\theta}_t+\frac{b_{max}-\hat{\theta_t}}{2} & \text{if } X_t = 1 \\ \hat{\theta}_t-\frac{\hat{\theta}_t-b_{min}}{2} & \text{if } X_t = 0 \end{array} \right\rbrace\end{split}\]

Parameters:

thetafloat: The initial ability level estimate.
item_bankItemBank: An ItemBank containing all items in the bank. This is necessary to capture the maximum and minimum difficulty levels required by the method.
correctbool: Boolean value indicating if the examinee has answered only correctly (True) or only incorrectly (False) up until now.

Returns:

float: A new estimation for \(\theta\) that avoids infinite values.

Compute the mean squared error (MSE) between two array-like objects.

The MSE is used when measuring the precision with which a computerized adaptive test estimates examinees abilities [Chang2001]. Lower MSE values indicate better estimation accuracy.

The value is computed as:

\[MSE = \frac{\sum_{i=1}^{N} (\hat{\theta}_i - \theta_{i})^2}{N}\]

where \(\hat{\theta}_i\) is examinee \(i\) estimated ability and \(\theta_i\) is examinee \(i\) actual ability.

Parameters:

actualnpt.ArrayLike: Array-like (list, tuple, or numpy array) containing the true ability values (float type).
predictednpt.ArrayLike: Array-like (list, tuple, or numpy array) containing the estimated ability values (float type).

Returns:

float: The mean squared error between the predicted values and actual values.

Raises:

ValueError: If actual and predicted vectors are not the same size.

catsim.cat.overlap_rate(exposure_rates: ndarray[tuple[Any, ...], dtype[floating]], test_size: int) → float[source]

Compute the test overlap rate.

An average measure of how much of the test two examinees take is equal [Bar10]. The overlap rate provides insight into test security: higher values indicate that examinees see more similar items, potentially compromising test security.

The overlap rate is given by:

\[T=\frac{N}{Q}S_{r}^2 + \frac{Q}{N}\]

where \(N\) is the bank size, \(Q\) is the test size, and \(S_{r}^2\) is the variance of exposure rates.

If, for example \(T = 0.5\), it means that the tests of two random examinees have 50% of equal items.

Parameters:

exposure_ratesnumpy.ndarray: A numpy array containing the exposure rate (proportion) of each item, where each value is between 0 and 1, representing the proportion of examinees who received that item.
test_sizeint: The number of items in a test.

Returns:

float: Test overlap rate between 0 and 1.

Raises:

ValueError: If exposure rates are not between 0 and 1, if test size is not positive, or if test size is larger than bank size.

catsim.cat.random_response_vector(size: int) → list[bool][source]

Generate a list of random boolean values of the given size.

This function is useful for testing and simulation purposes when you need a random response pattern.

Parameters:

sizeint: The size of the list to be generated. Must be non-negative.

Returns:

list[bool]: A list of random boolean values with equal probability of True and False.

Compute the root mean squared error (RMSE) between two array-like objects.

A common value used when measuring the precision with which a computerized adaptive test estimates examinees abilities [Bar10]. RMSE is in the same units as the ability scale, making it easier to interpret than MSE.

The value is computed as:

\[RMSE = \sqrt{\frac{\sum_{i=1}^{N} (\hat{\theta}_i - \theta_{i})^2}{N}}\]

where \(\hat{\theta}_i\) is examinee \(i\) estimated ability and \(\theta_i\) is examinee \(i\) actual ability.

Parameters:

actualnpt.ArrayLike: Array-like (list, tuple, or numpy array) containing the true ability values (float type).
predictednpt.ArrayLike: Array-like (list, tuple, or numpy array) containing the estimated ability values (float type).

Returns:

float: The root mean squared error between the predicted values and actual values.

Raises:

ValueError: If actual and predicted vectors are not the same size.

Computerized Adaptive Test-related Functions – catsim.cat

Computerized Adaptive Test-related Functions – `catsim.cat`