Validating a problem-based learning process assessment tool in a Nepalese medical school

Abstract

Introductions: The newly established Patan Academy of Health Sciences (PAHS) has incorporated the measurement of non-cognitive skills and behaviors into the summative assessment in the setting of problem based learning (PBL). This study was conducted to validate a PBL process assessment tool for PAHS.

Methods: A list of 72 items of student behaviors observable in PBL tutorials was compiled from literature review. They were categorized under ten broad dimensions consistent with predefined PAHS Graduate Attributes. A series of PBL project committee meetings and expert inputs refined the list of 72 items to 47 and categorized them under eight dimensions. These 47 items, each with a 4-point rating scale, formed the Tutor Assessment of Student Tool (TAS-Tool). Twenty-four trained faculty members used the TAS-Tool to evaluate the performance of 41 senior high school students in PBL tutorials. 

Results: The internal-consistency of the TAS-Tool was very high rona’s .. eoal of to inonsistent ites furter increased it to 0.975. Principal components analysis with varimax rotation applied to the remaining 45 items gave seven components and explained 69.47% of the variation between the components. These seven components (% variation) were: Immersed in the Tutorial Process (20.16%); Professional (12.71%); Communicator and Team Leader (11.25%); Critical Thinker (8.77%); Reflector (6.22%); Creative (5.95%), and Sensitive (4.41%).

Conclusions: TAS-Tool was found to be reliable and valid instrument deemed applicable in formative PBL process assessment at PAHS starting with the pioneer cohort of medical students. Further validation of TASTool through longitudinal study with PAHS students is required for summative purpose.

Keywords: factor analysis, problem based learning, summative assessment, tool validation, Nepal