Item Selection Methods – catsim.selection

All implemented classes in this module inherit from a base abstract class Selector. Simulator allows that a custom selector be used during the simulation, as long as it also inherits from Selector.

class catsim.selection.AStratBBlockSelector(test_size)

Bases: StratifiedSelector

Implementation of the \(\alpha\)-stratified selector with \(b\) blocking proposed by [Chang2001], in which the item bank is sorted in ascending order according to the items difficulty parameter and then separated into \(M\) strata, each stratum containing gradually higher average difficulty.

Each of the \(M\) strata is then again separated into \(K\) sub-strata (\(k\) being the test size), according to their discrimination. The final item bank is then ordered such that the first sub-strata of each strata forms the first strata of the new ordered item bank, and so on. This method tries to balance the distribution of both parameters between all strata, after perceiving that they are correlated.

Parameters:

test_size – the number of items the test contains. The selector uses this parameter to create the correct number of strata.

class catsim.selection.AStratSelector(test_size)

Bases: StratifiedSelector

Implementation of the \(\alpha\)-stratified selector proposed by [Chang99], in which the item bank is sorted in ascending order according to the items discrimination parameter and then separated into \(K\) strata (\(K\) being the test size), each stratum containing gradually higher average discrimination. The \(\alpha\)-stratified selector then selects the first non-administered item from stratum \(k\), in which \(k\) represents the position in the test of the current item the examinee is being presented.

Parameters:

test_size – the number of items the test contains. The selector uses this parameter to create the correct number of strata.

class catsim.selection.ClusterSelector(clusters: List[int], method: str = 'item_info', r_max: float = 1, r_control: str = 'passive')

Bases: Selector

Cluster-based Item Selection Method.

[Men15]

Meneghetti, D. R. (2015). Metolodogia de seleção de itens em testes adaptativos informatizados baseada em agrupamento por similaridade (Mestrado). Centro Universitário da FEI. Retrieved from https://www.researchgate.net/publication/283944553_Metodologia_de_selecao_de_itens_em_Testes_Adaptativos_Informatizados_baseada_em_Agrupamento_por_Similaridade

Parameters:

• clusters – a list containing item cluster memberships

• r_max – maximum exposure rate for items

• method –

  one of the available methods for cluster selection. Given the estimated theta value at each step:

  item_info: selects the cluster which has the item with maximum information;

  cluster_info: selects the cluster whose items sum of information is maximum;

  weighted_info: selects the cluster whose weighted sum of information is maximum. The weighted equals the number of items in the cluster;

• r_control – if passive and all items \(i\) in the selected cluster have \(r_i > r^{max}\), applies the item with maximum information; if aggressive, applies the item with smallest \(r\) value.

static avg_cluster_params(items: ndarray, c: List[int])

Returns the average values of item parameters by cluster

Parameters:

• items –

• c – a list containing clustering memeberships.

Returns:

a matrix containing the average values of each parameter by cluster. Lines are clusters, columns are parameters.

select(index: int | None = None, items: ndarray | None = None, administered_items: List[int] | None = None, est_theta: float | None = None, **kwargs) → int | None

Returns the index of the next item to be administered.

Parameters:

• index – the index of the current examinee in the simulator.

• items – a matrix containing item parameters in the format that catsim understands (see: catsim.cat.generate_item_bank())

• administered_items – a list containing the indexes of items that were already administered

• est_theta – a float containing the current estimated ability

Returns:

index of the next item to be applied.

static sum_cluster_infos(theta: float, items: ndarray, clusters: List[int]) → ndarray

Returns the sum of item information values, separated by cluster

Parameters:

• theta – an examinee’s \(\theta\) value

• items – a matrix containing item parameters in the format that catsim understands (see: catsim.cat.generate_item_bank())

• clusters – a list containing item cluster memberships, represented by integers

Returns:

array containing the sum of item information values for each cluster

static sum_cluster_params(items: ndarray, c: List[int])

Returns the sum of item parameter values for each cluster

Parameters:

• items – a matrix containing item parameters in the format that catsim understands (see: catsim.cat.generate_item_bank())

• c – a list containing clustering memeberships.

Returns:

a matrix containing the sum of each parameter by cluster. Lines are clusters, columns are parameters.

static weighted_cluster_infos(theta: float, items: ndarray, clusters: List[int]) → ndarray

Returns the weighted sum of item information values, separated by cluster. The weight is the number of items in each cluster.

Parameters:

• theta – an examinee’s \(\theta\) value

• items – a matrix containing item parameters in the format that catsim understands (see: catsim.cat.generate_item_bank())

• clusters – a list containing item cluster memberships, represented by integers

Returns:

array containing the sum of item information values for each cluster, divided by the number of items in each cluster

class catsim.selection.IntervalInfoSelector(interval: float | None = None)

Bases: Selector

A selector in which, at every step of the test, the item that maximizes the integral of the information function at a predetermined interval \(\delta\) above and below the current \(\hat\theta\) is chosen.

\[argmax_{i \in I} \int_{\hat\theta - \delta}^{\hat\theta + \delta}I_i(\hat\theta)\]

Parameters:

interval – the interval of the integral. If no interval is passed, the integral is computed from \([-\infty, \infty]\).

select(index: int | None = None, items: ndarray | None = None, administered_items: List[int] | None = None, est_theta: float | None = None, **kwargs) → int | None

Returns the index of the next item to be administered.

Parameters:

• index – the index of the current examinee in the simulator.

• items – a matrix containing item parameters in the format that catsim understands (see: catsim.cat.generate_item_bank())

• administered_items – a list containing the indexes of items that were already administered

• est_theta – a float containing the current estimated ability

Returns:

index of the next item to be applied or None if there are no more items in the item bank.

class catsim.selection.LinearSelector(indexes: List[int])

Bases: FiniteSelector

Selector that returns item indexes in a linear order, simulating a standard (non-adaptive) test.

Parameters:

indexes – the indexes of the items that will be returned in order

select(index: int | None = None, administered_items: List[int] | None = None, **kwargs) → int | None

Returns the index of the next item to be administered.

Parameters:

• index – the index of the current examinee in the simulator.

• administered_items – a list containing the indexes of items that were already administered

Returns:

index of the next item to be applied or None if there are no more items in the item bank.

class catsim.selection.MaxInfoBBlockSelector(test_size)

Bases: MaxInfoStratSelector

Implementation of the maximum information stratification with \(b\) blocking (MIS-B) selector proposed by [Bar06], in which the item bank is sorted in ascending order according to the items difficulty parameter and then separated into \(M\) strata, each stratum containing gradually higher average difficulty.

Each of the \(M\) strata is then again separated into \(K\) sub-strata (\(k\) being the test size), according to the items maximum information. The final item bank is then ordered such that the first sub-strata of each strata forms the first strata of the new ordered item bank, and so on. This method tries to balance the distribution of both parameters between all strata and works better than the \(a\)-stratified with \(b\) blocking method by [Chang2001] for the three-parameter logistic model of IRT, since item difficulty and maximum information are not positioned in the same place in the ability scale in 3PL. This may also apply, although not mentioned by the authors, for the 4PL.

Parameters:

test_size – the number of items the test contains. The selector uses this parameter to create the correct number of strata.

class catsim.selection.MaxInfoSelector(r_max: float = 1)

Bases: Selector

Selector that returns the first non-administered item with maximum information, given an estimated theta

Parameters:

r_max – maximum exposure rate for items

select(index: int | None = None, items: ndarray | None = None, administered_items: List[int] | None = None, est_theta: float | None = None, **kwargs) → int | None

Returns the index of the next item to be administered.

Parameters:

• index – the index of the current examinee in the simulator.

• items – a matrix containing item parameters in the format that catsim understands (see: catsim.cat.generate_item_bank())

• administered_items – a list containing the indexes of items that were already administered

• est_theta – a float containing the current estimated ability

Returns:

index of the next item to be applied or None if there are no more items in the item bank.

class catsim.selection.MaxInfoStratSelector(test_size)

Bases: StratifiedSelector

Implementation of the maximum information stratification (MIS) selector proposed by [Bar06], in which the item bank is sorted in ascending order according to the items maximum information and then separated into \(K\) strata (\(K\) being the test size), each stratum containing items with gradually higher maximum information. The MIS selector then selects the first non-administered item from stratum \(k\), in which \(k\) represents the position in the test of the current item the examinee is being presented.

This method claims to work better than the \(a\)-stratified method by [Chang99] for the three-parameter logistic model of IRT, since item difficulty and maximum information are not positioned in the same place in the ability scale in 3PL.

Parameters:

test_size – the number of items the test contains. The selector uses this parameter to create the correct number of strata.

class catsim.selection.RandomSelector(replace: bool = False)

Bases: Selector

Selector that randomly selects items for application.

Parameters:

replace – whether to select an item that has already been selected before for this examinee.

select(index: int | None = None, items: ndarray | None = None, administered_items: List[int] | None = None, **kwargs) → int | None

Returns the index of the next item to be administered.

Parameters:

• index – the index of the current examinee in the simulator.

• items – a matrix containing item parameters in the format that catsim understands (see: catsim.cat.generate_item_bank())

• administered_items – a list containing the indexes of items that were already administered

Returns:

index of the next item to be applied or None if there are no more items in the item bank.

class catsim.selection.RandomesqueSelector(bin_size)

Bases: Selector

Implementation of the randomesque selector proposed by [Kingsbury89], in which, at every step of the test, an item is randomly chosen from the \(n\) most informative items in the item bank, \(n\) being a predefined value (originally 5, but user-defined in this implementation)

Parameters:

bin_size – the number of most informative items to be taken into consideration when randomly selecting one of them.

select(index: int | None = None, items: ndarray | None = None, administered_items: List[int] | None = None, est_theta: float | None = None, **kwargs) → int | None

Returns the index of the next item to be administered.

Parameters:

• index – the index of the current examinee in the simulator.

• items – a matrix containing item parameters in the format that catsim understands (see: catsim.cat.generate_item_bank())

• administered_items – a list containing the indexes of items that were already administered

• est_theta – a float containing the current estimated ability

Returns:

index of the next item to be applied or None if there are no more items in the item bank.

class catsim.selection.StratifiedSelector(test_size, sort_once)

Bases: FiniteSelector

preprocess()

Override this method to initialize any static values the Simulable might use for the duration of the simulation. preprocess is called after a value is set for the simulator property. If a new value if attributed to simulator, this method is called again, guaranteeing that internal properties of the Simulable are re-initialized as necessary.

select(index: int | None = None, items: ndarray | None = None, administered_items: List[int] | None = None, **kwargs) → int | None

Returns the index of the next item to be administered.

Parameters:

• index – the index of the current examinee in the simulator.

• items – a matrix containing item parameters

• administered_items – a list containing the indexes of items that were already administered

Returns:

index of the next item to be applied or None if there are no more strata to get items from.

class catsim.selection.The54321Selector(test_size)

Bases: FiniteSelector

Implementation of the 5-4-3-2-1 selector proposed by [McBride83], in which, at each step \(k\) of a test of size \(K\), an item is chosen from a bin containing the \(K-k\) most informative items in the bank, given the current \(\hat\theta\). As the test progresses, the bin gets smaller and more informative items have a higher probability of being chosen by the end of the test, when the estimation of ‘\(\hat\theta\) is more precise. The 5-4-3-2-1 selector can be viewed as a specialization of the catsim.selection.RandomesqueSelector, in which the bin size of most informative items gets smaller as the test progresses.

Parameters:

test_size – the number of items the test contains. The selector uses this parameter to set the bin size

select(index: int | None = None, items: ndarray | None = None, administered_items: List[int] | None = None, est_theta: float | None = None, **kwargs) → int | None

Returns the index of the next item to be administered.

Parameters:

• index – the index of the current examinee in the simulator.

• items – a matrix containing item parameters in the format that catsim understands (see: catsim.cat.generate_item_bank())

• administered_items – a list containing the indexes of items that were already administered

• est_theta – a float containing the current estimated ability

Returns:

index of the next item to be applied or None if there are no more items in the item bank.

class catsim.selection.UrrySelector

Bases: Selector

Selector that returns the item whose difficulty parameter is closest to the examinee’s ability

select(index: int | None = None, items: ndarray | None = None, administered_items: List[int] | None = None, est_theta: float | None = None, **kwargs) → int | None

Returns the index of the next item to be administered.

Parameters:

• index – the index of the current examinee in the simulator.

• items – a matrix containing item parameters in the format that catsim understands (see: catsim.cat.generate_item_bank())

• administered_items – a list containing the indexes of items that were already administered

• est_theta – a float containing the current estimated ability

Returns:

index of the next item to be applied or None if there are no more items in the item bank.