Stopping Criteria – catsim.stopping

All implemented classes in this module inherit from a base abstract class BaseStopper. Simulator allows that a custom stopping criterion be used during the simulation, as long as it also inherits from BaseStopper.

Stopping criteria components for CAT.

class catsim.stopping.BaseStopper

Bases: Simulable, ABC

Base class for CAT stopping criteria.

Stoppers determine when a test should end based on specific criteria such as test length, measurement precision, or other conditions.

abstractmethod stop(index: int | None = None, **kwargs: Any) → bool

Check whether the test reached its stopping criterion for the given user.

Parameters:

indexint or None, optional

The index of the current examinee in the simulator. When used within a simulation, this parameter is provided automatically. When used standalone, other parameters may be provided via kwargs. Default is None.

**kwargsdict

Additional keyword arguments that specific Stopper implementations may require. Common arguments include:

• administered_items: Item parameters or indices that were administered

• theta: Current ability estimate

• item_bank: ItemBank for accessing item parameters

Returns:

bool

True if the test met its stopping criterion, False otherwise.

class catsim.stopping.ConfidenceIntervalStopper(interval_bounds: list[float], confidence: float = 0.95, min_items: int | None = None, max_items: int | None = None)

Bases: TestLengthStopper

Stopping criterion based on confidence interval falling within discrete ability intervals.

This stopper is designed for tests with discrete performance levels (e.g., letter grades A, B, C, D, F) defined by intervals on the ability scale. The test stops when the confidence interval for the examinee’s ability estimate falls entirely within one of these discrete intervals, indicating sufficient precision to classify the examinee.

This stopper also enforces optional minimum/maximum item constraints and stops when the item bank is exhausted (via BaseStopper).

Parameters:

interval_boundslist[float]

Sorted list of boundary points defining the discrete intervals on the ability scale. For example, [-2.0, -0.5, 0.5, 2.0] defines 5 intervals: (-inf, -2.0), [-2.0, -0.5), [-0.5, 0.5), [0.5, 2.0), [2.0, inf)

confidencefloat, optional

The confidence level for computing the confidence interval, must be between 0 and 1. Default is 0.95 (95% confidence).

min_itemsint or None, optional

Minimum number of items that must be administered before the test can stop. Default is None (no minimum).

max_itemsint or None, optional

Maximum number of items that can be administered. Default is None (no maximum).

Examples

>>> # Define grade boundaries: F (<-1), D [-1, 0), C [0, 1), B [1, 2), A (>=2)
>>> stopper = ConfidenceIntervalStopper([-1.0, 0.0, 1.0, 2.0], confidence=0.95)
>>> # Test stops when 95% CI is entirely within one grade interval

>>> # With minimum items constraint
>>> stopper = ConfidenceIntervalStopper(
...     [-1.0, 0.0, 1.0, 2.0], confidence=0.90, min_items=10
... )
>>> # Test cannot stop before 10 items, even if CI criterion is met

>>> # With maximum items constraint
>>> stopper = ConfidenceIntervalStopper(
...     [-1.0, 0.0, 1.0, 2.0], confidence=0.95, max_items=50
... )
>>> # Test stops at 50 items even if CI criterion is not met

property confidence: float

Get the confidence level.

Returns:

float

The confidence level used for computing confidence intervals.

property interval_bounds: list[float]

Get the interval boundary points.

Returns:

list[float]

The boundary points defining the discrete intervals.

class catsim.stopping.MinErrorStopper(min_error: float, min_items: int | None = None, max_items: int | None = None)

Bases: TestLengthStopper

Stopping criterion based on minimum standard error of estimation.

The test stops when the standard error of estimation (see catsim.irt.see()) falls below the specified threshold. This is commonly used in variable-length CATs to achieve a desired level of measurement precision.

This stopper also enforces optional minimum/maximum item constraints and stops when the item bank is exhausted (via BaseStopper).

Parameters:

min_errorfloat

The minimum standard error of estimation the test must achieve before stopping. Must be positive. Smaller values require more items for higher precision.

min_itemsint or None, optional

Minimum number of items that must be administered before the test can stop. Default is None (no minimum).

max_itemsint or None, optional

Maximum number of items that can be administered. Default is None (no maximum).

Examples

>>> # Stop when error < 0.3
>>> stopper = MinErrorStopper(0.3)

>>> # Stop when error < 0.3, but only after at least 10 items
>>> stopper = MinErrorStopper(0.3, min_items=10)

>>> # Stop when error < 0.3, or when 50 items reached
>>> stopper = MinErrorStopper(0.3, max_items=50)

>>> # Stop when error < 0.3, between 10 and 50 items
>>> stopper = MinErrorStopper(0.3, min_items=10, max_items=50)

property min_error: float

Get the minimum error threshold.

Returns:

float

The minimum error threshold.

class catsim.stopping.TestLengthStopper(min_items: int | None = None, max_items: int | None = None)

Bases: BaseStopper

Base class for stoppers with common min/max item constraints and bank exhaustion checks.

This class provides common functionality for stopping criteria including: - Minimum items requirement (test cannot stop before min_items are administered) - Maximum items constraint (test must stop when max_items are reached) - Item bank exhaustion detection (test stops when all items are used)

Subclasses must implement the _check_stopping_criterion method to define their specific stopping logic.

Parameters:

min_itemsint or None, optional

Minimum number of items that must be administered before the test can stop. If None, no minimum is enforced. Default is None.

max_itemsint or None, optional

Maximum number of items that can be administered. Test stops when this limit is reached regardless of other criteria. If None, no maximum is enforced. Default is None.

Notes

The stopping logic follows this priority order: 1. Stop if max_items reached (hard stop) 2. Stop if item bank exhausted (hard stop) 3. Do not stop if min_items not yet reached (regardless of other criteria) 4. Check the specific stopping criterion implemented by the subclass

property max_items: int | None

Get the maximum number of items.

Returns:

int or None

The maximum number of items, or None if no maximum is set.

property min_items: int | None

Get the minimum number of items.

Returns:

int or None

The minimum number of items, or None if no minimum is set.

stop(index: int | None = None, _item_bank: ItemBank | None = None, administered_items: ndarray[tuple[Any, ...], dtype[floating[Any]]] | None = None, theta: float | None = None, **kwargs: Any) → bool

Check whether the test should stop based on common constraints and specific criterion.

Parameters:

indexint or None, optional

The index of the current examinee. Default is None.

_item_bankItemBank or None, optional

The item bank being used. Default is None.

administered_itemsnpt.NDArray[numpy.floating[Any]] or None, optional

A matrix containing the parameters of items that were already administered. Default is None.

thetafloat or None, optional

An ability value. Default is None.

**kwargsdict

Additional keyword arguments passed to the specific stopping criterion.

Returns:

bool

True if the test should stop, False otherwise.

Raises:

ValueError

If required parameters are missing.