Bilişsel tanı modelinde geleneksel ve bilgisayarlı sınıflamalı test uygulamalarının psikometrik özelliklerinin karşılaştırılması
Abstract
Classroom and large scale measurement and evaluation applications of Turkey's education system is observed to be result to focus on results. Only if applications which provide chances for formative assessment are used more often within the context of measurement and evaluation applications, existing situation can be diagnosed via monitoring the learnings through the process; and, learning deficiencies can be resolved on time. Thanks to that, quality of education is thought to be enhanced. Cognitive Diagnostic Model (CDM) is a psychometric model that allows the assessment of students' weaknesses and strengths by means of the information it provides. Individuals in this model are assigned to some latent classes based on their item response patterns; and, thanks to that, information to identify how well test takers perform in related cognitive processes required to answer the items correctly is provided for researchers. Cognitive Diagnostic Computer Adaptive Test (CD-CAT) applications ensure the advantages of computerized adaptive tests and Cognitive Diagnosis Model which puts forward formative assessment by determining the learning deficiencies that are specific to individuals together. However, limitations of the CD-CAT in the application process create a need to look examine alternatives. Within this context, it was aimed in this study to investigate whether latent class results in which students were placed according to the results of Classified CAT (C-CAT) and Hierarchical CDM using a mathematics test developed by researchers is consistent. For this aim, a mathematics test consisting of 89 items was developed which had 5 forms measuring the four fundamental cognitive skills in mathematics number subject area, fractions sub-learning domain. These forms were applied to 1380 student of 6th-7th and 8th grades from 12 public schools which were selected from Ankara city. Latent class estimations were conducted via Hierarchical CDM to reveal the skills regarding the fractions learning domain using the data set obtained from this application. Later, with the help of the simulative studies produced from the same data, estimations were calculated with C-CAT which is an another psychometric approach used to assign individuals to latent classes; and, lastly, classification consistency between these two models were examined. Besides, detailed diagnostic outputs were reported both for the individuals and the group thanks to the advantages CDM provided. Based on the findings obtained from the analysis, it was found out when the number of people assigned to each latent class in CDM is studied that the most crowded group consisted of students who had no skills (0000) measured by the test. This group was occurred to be followed by the group with students who had three skills (1110) measured by the test; and the last group with students who had all skills (1111) measured by the test. This result was observed to repeat when it was studied based on the classroom levels (6-7 and 8. grades) separately. For C-CAT analysis, bias, RMSE and Average Test Length (ATL) were calculated using the simulative data based on the real test data for 12 different conditions which were designed by crossing 3 different test stopping rules and 4 different item selection rules (3*4). Considering these values (bias, RMSE and ATL) as criteria, conditions were compared to each other and the best fitted condition with the data was attempted to be determined. Within this context, Sequential Probability Ratio Test (SPRT) / Maximum Fisher Information (MFI) provided the best fit with regards to bias and RMSE while Condition Indeks (CI) provided the best fit for ATL. It was determined that Generalized Likelihood Ratio (GLR) and CI produced similar results to each other while SPRT had the worst result with regards to ATL. There found no significant differences between item selection methods. Classification consistency obtained from CDM and C-CAT was calculated to be (%52) when both analysis results were evaluated together. It was resulted that there was a medium level coherence between the classifications made based on two methods. Lastly, Diagnostic Result Reports based on CDM were designed for individuals selected from the sample and detailed outputs were presented regarding the individuals' cognitive development level. Proficiency descriptions for each latent class were formed. Furthermore, members of each latent class were specified in a detailed way for what they can and cannot do. Discussions regarding how two people with same total score could take place in different latent classes if the estimations were made with CDM and the advantages of presenting detailed feedbacks when using CDM were provided.