Stopping Criteria – `catsim.stopping`

All implemented classes in this module inherit from a base abstract class Stopper. Simulator allows that a custom stopping criterion be used during the simulation, as long as it also inherits from Stopper.

class catsim.stopping.CombinationStopper(stoppers: list[Stopper], strategy: CombinationStrategy = CombinationStrategy.OR)[source]

Bases: Stopper

Stopping criterion that combines multiple stoppers using AND or OR logic.

This stopper allows you to combine multiple stopping criteria in a flexible way. With OR logic (default), the test stops when ANY of the stoppers indicates stopping. With AND logic, the test stops only when ALL stoppers indicate stopping.

Parameters:

stopperslist[Stopper]: A list of Stopper instances to combine. Must contain at least one stopper.
strategyCombinationStrategy, optional: The combination strategy to use. Default is CombinationStrategy.OR.

Examples

>>> # Stop when either 30 items OR error < 0.4 (whichever comes first)
>>> stopper = CombinationStopper(
...     [MaxItemStopper(30), MinErrorStopper(0.4)],
...     strategy=CombinationStrategy.OR
... )

>>> # Stop only when BOTH 20 items AND error < 0.3 are achieved
>>> stopper = CombinationStopper(
...     [MaxItemStopper(20), MinErrorStopper(0.3)],
...     strategy=CombinationStrategy.AND
... )

>>> # Combine three stoppers: stop when any condition is met
>>> stopper = CombinationStopper(
...     [
...         MaxItemStopper(50),
...         MinErrorStopper(0.3),
...         ConfidenceIntervalStopper([-1.0, 0.0, 1.0], confidence=0.90)
...     ],
...     strategy=CombinationStrategy.OR
... )

stop(index: int | None = None, _item_bank: ItemBank | None = None, administered_items: ndarray[tuple[Any, ...], dtype[floating[Any]]] | None = None, theta: float | None = None, **kwargs: Any) → bool[source]

Check whether the combined stopping criterion is met.

Parameters:

indexint or None, optional: The index of the current examinee. Default is None.
administered_itemsnpt.NDArray[numpy.floating[Any]] or None, optional: A matrix containing the parameters of items that were already administered. Default is None.
thetafloat or None, optional: An ability value. Default is None.
**kwargsdict: Additional keyword arguments passed to individual stoppers.

Returns:

bool

If strategy is OR: True if ANY stopper indicates stopping
If strategy is AND: True if ALL stoppers indicate stopping

Notes

The CombinationStopper extracts data from the simulator (if needed) and calls all child stoppers in standalone mode with explicit parameters, so the child stoppers don’t need to depend on having a simulator reference.

property stoppers: list[Stopper]

Get the list of stoppers being combined.

Returns:

list[Stopper]: The list of Stopper instances.

property strategy: CombinationStrategy

Get the combination strategy.

Returns:

CombinationStrategy: The combination strategy (OR or AND).

class catsim.stopping.CombinationStrategy(*values)[source]

Bases: Enum

Strategy for combining multiple stopping criteria.

Attributes:

ORstr: Test stops when ANY of the stoppers indicates stopping (logical OR). This is useful when you want the test to end as soon as any condition is met.
ANDstr: Test stops when ALL of the stoppers indicate stopping (logical AND). This is useful when you want the test to continue until all conditions are met.

class catsim.stopping.ConfidenceIntervalStopper(interval_bounds: list[float], confidence: float = 0.95)[source]

Bases: Stopper

Stopping criterion based on confidence interval falling within discrete ability intervals.

This stopper is designed for tests with discrete performance levels (e.g., letter grades A, B, C, D, F) defined by intervals on the ability scale. The test stops when the confidence interval for the examinee’s ability estimate falls entirely within one of these discrete intervals, indicating sufficient precision to classify the examinee.

The figure below illustrates the concept with an example where an examinee’s ability is estimated at θ = 1.0 with a confidence interval that falls entirely within the grade B interval [0.5, 1.5), causing the test to stop.

(Source code, png, hires.png, pdf)

_images/stopping-1.png — Fig. 7 Confidence interval-based stopping. The shaded area shows the 90% confidence interval around the estimated ability (θ = 1.0). Since the entire confidence interval falls within the grade B boundaries, the test stops.

Parameters:

interval_boundslist[float]: Sorted list of boundary points defining the discrete intervals on the ability scale. For example, [-2.0, -0.5, 0.5, 2.0] defines 5 intervals: (-inf, -2.0), [-2.0, -0.5), [-0.5, 0.5), [0.5, 2.0), [2.0, inf)
confidencefloat, optional: The confidence level for computing the confidence interval, must be between 0 and 1. Default is 0.95 (95% confidence).

Examples

>>> # Define grade boundaries: F (<-1), D [-1, 0), C [0, 1), B [1, 2), A (>=2)
>>> stopper = ConfidenceIntervalStopper([-1.0, 0.0, 1.0, 2.0], confidence=0.95)
>>> # Test stops when 95% CI is entirely within one grade interval

property confidence: float

Get the confidence level.

Returns:

float: The confidence level used for computing confidence intervals.

property interval_bounds: list[float]

Get the interval boundary points.

Returns:

list[float]: The boundary points defining the discrete intervals.

stop(index: int | None = None, _item_bank: ItemBank | None = None, administered_items: ndarray[tuple[Any, ...], dtype[floating[Any]]] | None = None, theta: float | None = None, **kwargs: Any) → bool[source]

Check whether the confidence interval falls entirely within a discrete interval.

Parameters:

indexint or None, optional: The index of the current examinee. Default is None.
administered_itemsnpt.NDArray[numpy.floating[Any]] or None, optional: A matrix containing the parameters of items that were already administered. Default is None.
thetafloat or None, optional: An ability value to which the confidence interval will be computed. Default is None.
**kwargsdict: Additional keyword arguments. Not used by this method.

Returns:

bool: True if the confidence interval for the ability estimate falls entirely within one of the discrete intervals, False otherwise.

Raises:

ValueError: If required parameters are missing.

class catsim.stopping.ItemBankLengthStopper[source]

Bases: Stopper

Stopping criterion based on exhausting the entire item bank.

The test stops when all items in the item bank have been administered to the examinee. This is useful for ensuring that an examinee never runs out of items, or for creating tests that must use all available items.

This stopper is typically used in combination with other stoppers (via CombinationStopper with OR strategy) to prevent item exhaustion errors.

Examples

>>> # Stop when item bank is exhausted (safety mechanism)
>>> stopper = ItemBankLengthStopper()

>>> # Combine with other criteria: stop at 30 items OR when bank exhausted
>>> from catsim.stopping import CombinationStopper, CombinationStrategy
>>> stopper = CombinationStopper(
...     [MaxItemStopper(30), ItemBankLengthStopper()],
...     strategy=CombinationStrategy.OR
... )

stop(index: int | None = None, _item_bank: ItemBank | None = None, administered_items: ndarray[tuple[Any, ...], dtype[floating[Any]]] | None = None, **kwargs: Any) → bool[source]

Check whether all items in the item bank have been administered.

Parameters:

indexint or None, optional: The index of the current examinee. Default is None.
_item_bankItemBank or None, optional: The item bank being used. Default is None.
administered_itemsnpt.NDArray[numpy.floating[Any]] or None, optional: A matrix containing the parameters of items that were already administered. Default is None.
**kwargsdict: Additional keyword arguments. Not used by this method.

Returns:

bool: True if all items in the item bank have been administered, False otherwise.

Raises:

ValueError: If required parameters are missing.

class catsim.stopping.MaxItemStopper(max_itens: int)[source]

Bases: Stopper

Stopping criterion based on maximum number of items in a test.

The test stops when the specified maximum number of items has been administered. This is the most common stopping criterion in fixed-length CATs.

Parameters:

max_itensint: The maximum number of items to be administered in the test.

property max_itens: int

Get the maximum number of items the Stopper is configured to administer.

Returns:

int: The maximum number of items the Stopper is configured to administer.

stop(index: int | None = None, _item_bank: ItemBank | None = None, administered_items: ndarray[tuple[Any, ...], dtype[floating[Any]]] | None = None, **kwargs: Any) → bool[source]

Check whether the test reached its stopping criterion for the given user.

Parameters:

indexint or None, optional: The index of the current examinee. Default is None.
administered_itemsnpt.NDArray[numpy.floating[Any]] or None, optional: A matrix containing the parameters of items that were already administered. Default is None.
**kwargsdict: Additional keyword arguments. Not used by this method.

Returns:

bool: True if the test met its stopping criterion (maximum items reached), False otherwise.

Raises:

ValueError: If more items than permitted were administered, or if required parameters are missing.

class catsim.stopping.MinErrorStopper(min_error: float)[source]

Bases: Stopper

Stopping criterion based on minimum standard error of estimation.

The test stops when the standard error of estimation (see catsim.irt.see()) falls below the specified threshold. This is commonly used in variable-length CATs to achieve a desired level of measurement precision.

Parameters:

min_errorfloat: The minimum standard error of estimation the test must achieve before stopping. Must be positive. Smaller values require more items for higher precision.

stop(index: int | None = None, _item_bank: ItemBank | None = None, administered_items: ndarray[tuple[Any, ...], dtype[floating[Any]]] | None = None, theta: float | None = None, **kwargs: Any) → bool[source]

Check whether the test reached its stopping criterion.

Parameters:

indexint or None, optional: The index of the current examinee. Default is None.
administered_itemsnpt.NDArray[numpy.floating[Any]] or None, optional: A matrix containing the parameters of items that were already administered. Default is None.
thetafloat or None, optional: An ability value to which the error will be computed. Default is None.
**kwargsdict: Additional keyword arguments. Not used by this method.

Returns:

bool: True if the test met its stopping criterion (standard error below minimum), False otherwise.

Raises:

ValueError: If required parameters are missing.

Stopping Criteria – catsim.stopping

Stopping Criteria – `catsim.stopping`