Computerized Adaptive Testing Simulation – catsim.simulation

Module containing functions relevant to the process of simulating the application of adaptive tests. Most of this module is based on the work of [Bar10].

class catsim.simulation.Estimator[source]

Base class for proficiency estimators

abstract estimate(index: int)float[source]
Returns the theta value that minimizes the negative log-likelihood function, given the current state of the

test for the given examinee.

Parameters

index – index of the current examinee in the simulator

Returns

the current \(\hat\theta\)

class catsim.simulation.FiniteSelector(test_size)[source]

Base class representing a CAT item selector.

class catsim.simulation.Initializer[source]

Base class for CAT initializers

abstract initialize(index: int)float[source]

Selects an examinee’s initial \(\theta\) value

Parameters

index – the index of the current examinee

Returns

examinee’s initial \(\theta\) value

class catsim.simulation.Selector[source]

Base class representing a CAT item selector.

abstract select(index: int)int[source]

Returns the index of the next item to be administered.

Parameters

index – the index of the current examinee in the simulator.

Returns

index of the next item to be applied or None if there are no more items to be presented.

class catsim.simulation.Simulable[source]

Base class representing one of the Simulator components that will receive a reference back to it.

preprocess()[source]

Override this method to initialize any static values the Simulable might use for the duration of the simulation. preprocess is called after a value is set for the simulator property. If a new value if attributed to simulator, this method is called again, guaranteeing that internal properties of the Simulable are re-initialized as necessary.

class catsim.simulation.Simulator(items: numpy.ndarray, examinees, initializer: catsim.simulation.Initializer = None, selector: catsim.simulation.Selector = None, estimator: catsim.simulation.Estimator = None, stopper: catsim.simulation.Stopper = None)[source]

Class representing the simulator. It gathers several objects that describe the full simulation process and simulates one or more computerized adaptive tests

Parameters
  • items – a matrix containing item parameters

  • examinees – an integer with the number of examinees, whose real \(\theta\) values will be sampled from a normal distribution; or a :py:type:list containing said \(\theta_0\) values

property administered_items

List of lists containing the indexes of items administered to each examinee during the simulation.

property bias

Bias between the estimated and true proficiencies. This property is only available after simulate() has been successfully called. For more information on estimation bias, see catsim.cat.bias()

property duration

Duration of the simulation, in seconds.

property estimations

List of lists containing all estimated \(\hat\theta\) values for all examinees during each step of the test.

property examinees

List containing examinees true proficiency values (\(\theta\)).

property items

Item matrix used by the simulator. If the simulation already occurred, a column containing item exposure rates will be added to the matrix.

property latest_estimations

Final estimated \(\hat\theta\) values for all examinees.

property mse

Mean-squared error between the estimated and true proficiencies. This property is only available after simulate() has been successfully called. For more information on the mean-squared error of estimation, see catsim.cat.mse()

property overlap_rate

Overlap rate of the test, if it is of finite length.

property response_vectors

List of boolean lists containing the examinees answers to all items.

property rmse

Root mean-squared error between the estimated and true proficiencies. This property is only available after simulate() has been successfully called. For more information on the root mean-squared error of estimation, see catsim.cat.rmse()

simulate(initializer: catsim.simulation.Initializer = None, selector: catsim.simulation.Selector = None, estimator: catsim.simulation.Estimator = None, stopper: catsim.simulation.Stopper = None, verbose: bool = False)[source]

Simulates a computerized adaptive testing application to one or more examinees

Parameters
  • initializer – an initializer that selects examinees \(\theta_0\)

  • selector – a selector that selects new items to be presented to examinees

  • estimator – an estimator that reestimates examinees proficiencies after each item is applied

  • stopper – an object with a stopping criteria for the test

  • verbose – whether to periodically print a message regarding the progress of the simulation. Good for longer simulations.

>>> from catsim.initialization import RandomInitializer
>>> from catsim.selection import MaxInfoSelector
>>> from catsim.estimation import HillClimbingEstimator
>>> from catsim.stopping import MaxItemStopper
>>> from catsim.simulation import Simulator
>>> from catsim.cat import generate_item_bank
>>> initializer = RandomInitializer()
>>> selector = MaxInfoSelector()
>>> estimator = HillClimbingEstimator()
>>> stopper = MaxItemStopper(20)
>>> Simulator(generate_item_bank(100), 10).simulate(initializer, selector, estimator, stopper)
class catsim.simulation.Stopper[source]

Base class for CAT stop criterion

abstract stop(index: int)bool[source]

Checks whether the test reached its stopping criterion for the given user

Parameters

index – the index of the current examinee

Returns

True if the test met its stopping criterion, else False