Computerized Adaptive Testing Simulation – `catsim.simulation`¶

Module containing functions relevant to the process of simulating the application of adaptive tests.

Most of this module is based on the work of [Bar10].

Class representing the CAT simulator.

The Simulator gathers several objects that describe the full simulation process (initializer, selector, estimator, stopper) and simulates one or more computerized adaptive tests.

Parameters:

item_bankItemBank or numpy.ndarray: An ItemBank object containing item parameters. If a numpy.ndarray is provided, it will be automatically converted to an ItemBank.
examineesint or npt.ArrayLike: Either an integer with the number of examinees (whose real \(\theta\) values will be sampled from a normal distribution), or an array-like (list, tuple, or numpy array) containing the examinees’ true \(\theta\) values (float type).
initializerBaseInitializer or None, optional: BaseInitializer to use during the simulation. Default is None.
selectorSelector or None, optional: Selector to use during the simulation. Default is None.
estimatorEstimator or None, optional: Estimator to use during the simulation. Default is None.
stopperStopper or None, optional: Stopper to use during the simulation. Default is None.
seedint, optional: Seed used by the numpy random number generator during the simulation procedure. Default is 0.

property administered_items: list[list[int]]¶: A list of lists with the indexes of items administered to each examinee during the simulation.

property bias: float¶

Get the bias between the estimated and true abilities.

This property is only available after simulate() has been successfully called. For more information on estimation bias, see catsim.cat.bias()

property duration: float¶: Duration of the simulation, in seconds.

property estimations: list[list[float]]¶: A list of lists with all estimated \(\hat\theta\) values for all examinees during each step of the test.

property estimator: BaseEstimator | None¶

Get the estimator used during the simulation.

Returns:

Estimator or None: The estimator used during the simulation.

property examinees: ndarray[tuple[Any, ...], dtype[floating[Any]]]¶: :py:type:numpy.ndarray containing examinees true ability values (\(\theta\)).

property initializer: BaseInitializer | None¶

Get the initializer used during the simulation.

Returns:

BaseInitializer or None: The initializer used during the simulation.

property item_bank: ItemBank¶

ItemBank used by the simulator.

Returns:

ItemBank: The ItemBank containing all item parameters and exposure rates.

property items: ndarray[tuple[Any, ...], dtype[floating[Any]]]¶

Item matrix used by the simulator (for backward compatibility).

Returns:

npt.NDArray[numpy.floating[Any]]: The underlying item parameter matrix from the ItemBank.

property latest_estimations: list[float]¶: Final estimated \(\hat\theta\) values for all examinees.

property mse: float¶

Get the mean-squared error between the estimated and true abilities.

This property is only available after simulate() has been successfully called. For more information on the mean-squared error of estimation, see catsim.cat.mse().

property overlap_rate: float¶: Overlap rate of the test, if it is of finite length.

property response_vectors: list[list[bool]]¶: List of boolean lists containing the examinees answers to all items.

property rmse: float¶

Get the root mean-squared error between the estimated and true abilities.

This property is only available after simulate() has been successfully called. For more information on the root mean-squared error of estimation, see catsim.cat.rmse().

property rng: Generator¶: Get the random number generator used by the simulator.

property selector: BaseSelector | None¶

Get the selector used during the simulation.

Returns:

Selector or None: The selector used during the simulation.

simulate(initializer: BaseInitializer | None = None, selector: BaseSelector | None = None, estimator: BaseEstimator | None = None, stopper: BaseStopper | None = None, verbose: bool = False) → None[source]¶

Simulate a computerized adaptive testing application to one or more examinees.

The simulation process iterates through each examinee, initializing their ability, selecting items, recording responses, estimating abilities, and stopping when the criterion is met.

Parameters:

initializerInitializer or None, optional: An initializer that selects examinees’ initial \(\theta_0\). Default is None.
selectorSelector or None, optional: A selector that selects new items to be presented to examinees. Default is None.
estimatorEstimator or None, optional: An estimator that reestimates examinees abilities after each item is applied. Default is None.
stopperStopper or None, optional: An object with a stopping criterion for the test. Default is None.
verbosebool, optional: Whether to periodically print a message regarding the progress of the simulation. Good for longer simulations. Default is False.

Raises:

ValueError: If any of initializer, selector, estimator, or stopper is None (either passed or from constructor).

Examples

>>> from catsim.initialization import RandomInitializer
>>> from catsim.selection import MaxInfoSelector
>>> from catsim.estimation import NumericalSearchEstimator
>>> from catsim.stopping import MinErrorStopper
>>> from catsim.simulation import Simulator
>>> from catsim.item_bank import ItemBank
>>> initializer = RandomInitializer()
>>> selector = MaxInfoSelector()
>>> estimator = NumericalSearchEstimator()
>>> stopper = MinErrorStopper(0.4, max_items=20)
>>> Simulator(ItemBank.generate_item_bank(100), 10).simulate(initializer, selector, estimator, stopper)

property stopper: BaseStopper | None¶

Get the stopper used during the simulation.

Returns:

Stopper or None: The stopper used during the simulation.

Computerized Adaptive Testing Simulation – catsim.simulation¶

Computerized Adaptive Testing Simulation – `catsim.simulation`¶