Computerized Adaptive Testing Simulation – catsim.simulation

Module containing functions relevant to the process of simulating the application of adaptive tests. Most of this module is based on the work of [Bar10].

class catsim.simulation.Estimator[source]

Base class for proficiency estimators

estimate(index: int) → float[source]
Returns the theta value that minimizes the negative log-likelihood function, given the current state of the
test for the given examinee.
Return type:float
Parameters:index (int) – index of the current examinee in the simulator
Returns:the current \(\hat\theta\)
class catsim.simulation.FiniteSelector(test_size)[source]

Base class representing a CAT item selector.

class catsim.simulation.Initializer[source]

Base class for CAT initializers

initialize(index: int) → float[source]

Selects an examinee’s initial \(\theta\) value

Return type:float
Parameters:index (int) – the index of the current examinee
Returns:examinee’s initial \(\theta\) value
class catsim.simulation.Selector[source]

Base class representing a CAT item selector.

select(index: int) → int[source]

Returns the index of the next item to be administered.

Return type:int
Parameters:index (int) – the index of the current examinee in the simulator.
Returns:index of the next item to be applied or None if there are no more items to be presented.
class catsim.simulation.Simulable[source]

Base class representing one of the Simulator components that will receive a reference back to it.

preprocess()[source]

Override this method to initialize any static values the Simulable might use for the duration of the simulation. preprocess is called after a value is set for the simulator property. If a new value if attributed to simulator, this method is called again, guaranteeing that internal properties of the Simulable are re-initialized as necessary.

class catsim.simulation.Simulator(items: numpy.ndarray, examinees, initializer: catsim.simulation.Initializer = None, selector: catsim.simulation.Selector = None, estimator: catsim.simulation.Estimator = None, stopper: catsim.simulation.Stopper = None)[source]

Class representing the simulator. It gathers several objects that describe the full simulation process and simulates one or more computerized adaptive tests

Parameters:
  • items – a matrix containing item parameters
  • examinees – an integer with the number of examinees, whose real \(\theta\) values will be sampled from a normal distribution; or a :py:type:list containing said \(\theta_0\) values
administered_items

List of lists containing the indexes of items administered to each examinee during the simulation.

bias

Bias between the estimated and true proficiencies. This property is only available after simulate() has been successfully called. For more information on estimation bias, see catsim.cat.bias()

duration

Duration of the simulation, in milliseconds.

estimations

List of lists containing all estimated \(\hat\theta\) values for all examinees during each step of the test.

examinees

List containing examinees true proficiency values (\(\theta\)).

items

Item matrix used by the simulator. If the simulation already occurred, a column containing item exposure rates will be added to the matrix.

latest_estimations

Final estimated \(\hat\theta\) values for all examinees.

mse

Mean-squared error between the estimated and true proficiencies. This property is only available after simulate() has been successfully called. For more information on the mean-squared error of estimation, see catsim.cat.mse()

overlap_rate

Overlap rate of the test, if it is of finite length.

response_vectors

List of boolean lists containing the examinees answers to all items.

rmse

Root mean-squared error between the estimated and true proficiencies. This property is only available after simulate() has been successfully called. For more information on the root mean-squared error of estimation, see catsim.cat.rmse()

simulate(initializer: catsim.simulation.Initializer = None, selector: catsim.simulation.Selector = None, estimator: catsim.simulation.Estimator = None, stopper: catsim.simulation.Stopper = None, verbose: bool = False)[source]

Simulates a computerized adaptive testing application to one or more examinees

Parameters:
  • initializer (Initializer) – an initializer that selects examinees \(\theta_0\)
  • selector (Selector) – a selector that selects new items to be presented to examinees
  • estimator (Estimator) – an estimator that reestimates examinees proficiencies after each item is applied
  • stopper (Stopper) – an object with a stopping criteria for the test
  • verbose (bool) – whether to periodically print a message regarding the progress of the simulation. Good for longer simulations.
>>> from catsim.initialization import RandomInitializer
>>> from catsim.selection import MaxInfoSelector
>>> from catsim.estimation import HillClimbingEstimator
>>> from catsim.stopping import MaxItemStopper
>>> from catsim.simulation import Simulator
>>> from catsim.cat import generate_item_bank
>>> initializer = RandomInitializer()
>>> selector = MaxInfoSelector()
>>> estimator = HillClimbingEstimator()
>>> stopper = MaxItemStopper(20)
>>> Simulator(generate_item_bank(100), 10).simulate(initializer, selector, estimator, stopper)
class catsim.simulation.Stopper[source]

Base class for CAT stop criterion

stop(index: int) → bool[source]

Checks whether the test reached its stopping criterion for the given user

Return type:bool
Parameters:index (int) – the index of the current examinee
Returns:True if the test met its stopping criterion, else False