Evaluating Assessment Instruments for Measuring Students’ Higher-Order Thinking Skills in Heat and Temperature Concepts

Abstract

This study aimed to develop a higher-order thinking skills (HOTS)–based test instrument on temperature and heat topics and to examine its initial quality and feasibility as an assessment tool. The research employed a research and development (R&D) approach using the Borg and Gall model, which included stages of potential problem identification, information collection, product design, expert validation, product revision, and limited field testing. Expert validation using Aiken’s V yielded coefficients ranging from 0.82 to 0.94, indicating strong content validity. The limited field trial involved 32 ninth-grade students, and data analysis was conducted using Microsoft Excel and IBM SPSS Statistics 25. Empirical validity testing showed that 23 out of 40 items (57.5%) met the validity criterion, while the remaining items required revision or elimination. Reliability analysis using the KR-20 formula produced a coefficient of 0.776, indicating high internal consistency. Analysis of item difficulty revealed that 10 items (25%) were classified as difficult and 30 items (75%) as moderately difficult. Furthermore, the discriminating power analysis indicated that only 10 items (25%) demonstrated good discrimination, while 15 items (37.5%) fell into the sufficient category, suggesting that several items require further refinement to better differentiate students’ ability levels. Overall, the findings indicate that the developed HOTS-based test instrument demonstrates preliminary quality and conditional feasibility for measuring students’ higher-order thinking skills in junior high school science learning on temperature and heat topics. However, substantial revisions—particularly to improve item discrimination and increase the proportion of valid items—are necessary before the instrument can be recommended for broader implementation.