Computerized Adaptive Testrelated Functions – catsim.cat
¶
Functions used specifically during the application/simulation of computerized adaptive tests.

catsim.cat.
bias
(actual: list, predicted: list)[source]¶ Calculates the test bias, an evaluation criterion for computerized adaptive test methodolgies [Chang2001]. The value is calculated by:
\[Bias = \frac{\sum_{i=1}^{N} (\hat{\theta}_i  \theta_{i})}{N}\]where \(\hat{\theta}_i\) is examinee \(i\) estimated proficiency and \(\hat{\theta}_i\) is examinee \(i\) actual proficiency.
Parameters:  actual (
list
) – a list or 1D numpy array containing the true proficiency values  predicted (
list
) – a list or 1D numpy array containing the estimated proficiency values
Returns: the bias between the predicted values and actual values.
 actual (

catsim.cat.
dodd
(theta: float, items: numpy.ndarray, correct: bool) → float[source]¶ Method proposed by [Dod90] for the reestimation of \(\hat{\theta}\) when the response vector is composed entirely of 1s or 0s.
\[\begin{split}\hat{\theta}_{t+1} = \left\lbrace \begin{array}{ll} \hat{\theta}_t+\frac{b_{max}\hat{\theta_t}}{2} & \text{if } X_t = 1 \\ \hat{\theta}_t\frac{\hat{\theta}_tb_{min}}{2} & \text{if } X_t = 0 \end{array} \right\rbrace\end{split}\]Return type: float
Parameters:  theta (
float
) – the initial proficiency level  items (
ndarray
) – a numpy array containing the parameters of the items in the database. This is necessary to capture the maximum and minimum difficulty levels necessary for the method.  correct (
bool
) – a boolean value informing whether or not the examinee correctly answered the current item.
Returns: a new estimation for \(\theta\)
 theta (

catsim.cat.
generate_item_bank
(n: int, itemtype: str = '4PL', corr: float = 0)[source]¶ Generate a synthetic item bank whose parameters approximately follow realworld parameters, as proposed by [Bar10].
Item parameters are extracted from the following probability distributions:
 discrimination: \(N(1.2, 0.25)\)
 difficulty: \(N(0, 1)\)
 pseudoguessing: \(N(0.25, 0.02)\)
 upper asymptote: \(U(0.94, 1)\)
Parameters:  n (
int
) – how many items are to be generated  itemtype (
str
) – either1PL
,2PL
,3PL
or4PL
for the one, two, three or fourparameter logistic model  corr (
float
) – the correlation between item discrimination and difficulty. Ifitemtype == '1PL'
, it is ignored.
Returns: an
n x 4
numerical matrix containing item parametersReturn type: >>> generate_item_bank(5, '1PL') >>> generate_item_bank(5, '2PL') >>> generate_item_bank(5, '3PL') >>> generate_item_bank(5, '4PL') >>> generate_item_bank(5, '4PL', corr=0)

catsim.cat.
mse
(actual: list, predicted: list)[source]¶ Mean squared error, a value used when measuring the precision with which a computerized adaptive test estimates examinees proficiencies [Chang2001]. The value is calculated by:
\[MSE = \frac{\sum_{i=1}^{N} (\hat{\theta}_i  \theta_{i})^2}{N}\]where \(\hat{\theta}_i\) is examinee \(i\) estimated proficiency and \(\hat{\theta}_i\) is examinee \(i\) actual proficiency.
Parameters:  actual (
list
) – a list or 1D numpy array containing the true proficiency values  predicted (
list
) – a list or 1D numpy array containing the estimated proficiency values
Returns: the mean squared error between the predicted values and actual values.
 actual (

catsim.cat.
overlap_rate
(items: numpy.ndarray, test_size: int) → float[source]¶ Test overlap rate, an average measure of how much of the test two examinees take is equal [Bar10]. It is given by:
\[T=\frac{N}{Q}S_{r}^2 + \frac{Q}{N}\]If, for example \(T = 0.5\), it means that the tests of two random examinees have 50% of equal items.
Return type: float
Parameters:  items (
ndarray
) – a matrix containing, in the 4th column, the number of times each item was used in the tests.  test_size (
int
) – an integer informing the number of items in a test.
Returns: test overlap rate.
 items (

catsim.cat.
rmse
(actual: list, predicted: list)[source]¶ Root mean squared error, a common value used when measuring the precision with which a computerized adaptive test estimates examinees proficiencies [Bar10]. The value is calculated by:
\[RMSE = \sqrt{\frac{\sum_{i=1}^{N} (\hat{\theta}_i  \theta_{i})^2}{N}}\]where \(\hat{\theta}_i\) is examinee \(i\) estimated proficiency and \(\hat{\theta}_i\) is examinee \(i\) actual proficiency.
Parameters:  actual (
list
) – a list or 1D numpy array containing the true proficiency values  predicted (
list
) – a list or 1D numpy array containing the estimated proficiency values
Returns: the root mean squared error between the predicted values and actual values.
 actual (