Computerized Adaptive Testing Simulation – catsim.simulation
Module containing functions relevant to the process of simulating the application of adaptive tests. Most of this module is based on the work of [Bar10].
- class catsim.simulation.Estimator(verbose: bool = False)[source]
Base class for ability estimators
- property avg_evaluations
Average number of function evaluations for all tests the estimator has been used
- Returns:
average number of function evaluations
- property calls
How many times the estimator has been called to maximize/minimize the log-likelihood function
- Returns:
number of times the estimator has been called to maximize/minimize the log-likelihood function
- abstract estimate(index: int) float [source]
- Returns the theta value that minimizes the negative log-likelihood function, given the current state of the
test for the given examinee.
- Parameters:
index – index of the current examinee in the simulator
- Returns:
the current \(\hat\theta\)
- property evaluations
Total number of times the estimator has evaluated the log-likelihood function during its existence
- Returns:
number of function evaluations
- class catsim.simulation.FiniteSelector(test_size)[source]
Base class representing a CAT item selector.
- class catsim.simulation.Simulable[source]
Base class representing one of the Simulator components that will receive a reference back to it.
- preprocess()[source]
Override this method to initialize any static values the Simulable might use for the duration of the simulation. preprocess is called after a value is set for the simulator property. If a new value if attributed to simulator, this method is called again, guaranteeing that internal properties of the Simulable are re-initialized as necessary.
- class catsim.simulation.Simulator(items: ndarray, examinees: int | list | ndarray, initializer: Initializer | None = None, selector: Selector | None = None, estimator: Estimator | None = None, stopper: Stopper | None = None)[source]
Class representing the simulator. It gathers several objects that describe the full simulation process and simulates one or more computerized adaptive tests
- Parameters:
items – a matrix containing item parameters
examinees – an integer with the number of examinees, whose real \(\theta\) values will be sampled from a normal distribution; or a :py:type:list containing said \(\theta_0\) values
- property administered_items: list
List of lists containing the indexes of items administered to each examinee during the simulation.
- property bias: float
Bias between the estimated and true abilities. This property is only available after
simulate()
has been successfully called. For more information on estimation bias, seecatsim.cat.bias()
- property estimations: list
List of lists containing all estimated \(\hat\theta\) values for all examinees during each step of the test.
- property examinees: ndarray
:py:type:numpy.ndarray containing examinees true ability values (\(\theta\)).
- property items: ndarray
Item matrix used by the simulator. If the simulation already occurred, a column containing item exposure rates will be added to the matrix.
- property mse: float
Mean-squared error between the estimated and true abilities. This property is only available after
simulate()
has been successfully called. For more information on the mean-squared error of estimation, seecatsim.cat.mse()
- property response_vectors: list
List of boolean lists containing the examinees answers to all items.
- property rmse: float
Root mean-squared error between the estimated and true abilities. This property is only available after
simulate()
has been successfully called. For more information on the root mean-squared error of estimation, seecatsim.cat.rmse()
- simulate(initializer: Initializer | None = None, selector: Selector | None = None, estimator: Estimator | None = None, stopper: Stopper | None = None, verbose: bool = False)[source]
Simulates a computerized adaptive testing application to one or more examinees
- Parameters:
initializer – an initializer that selects examinees \(\theta_0\)
selector – a selector that selects new items to be presented to examinees
estimator – an estimator that reestimates examinees abilities after each item is applied
stopper – an object with a stopping criteria for the test
verbose – whether to periodically print a message regarding the progress of the simulation. Good for longer simulations.
>>> from catsim.initialization import RandomInitializer >>> from catsim.selection import MaxInfoSelector >>> from catsim.estimation import NumericalSearchEstimator >>> from catsim.stopping import MaxItemStopper >>> from catsim.simulation import Simulator >>> from catsim.cat import generate_item_bank >>> initializer = RandomInitializer() >>> selector = MaxInfoSelector() >>> estimator = NumericalSearchEstimator() >>> stopper = MaxItemStopper(20) >>> Simulator(generate_item_bank(100), 10).simulate(initializer, selector, estimator, stopper)