The policy says predefined difficulty levels for questions in all professional exams have been established, with each paper comprising 20% hard questions, 20% easy questions, and 60% moderate questions.
LAHORE: (UrduPoint/Pakistan Point News-Oct 21st, 2023) With an aim to standardize the question paper setting process in MBBS and BDS professional examinations, the University of Health Sciences (UHS) has formulated new policy guidelines and standard operating procedures for question bank development.
These guidelines which have been named as the 'Examiners Toolkit', are prepared by the UHS Committee for Conduct of Examinations (CCE) and will be implemented from annual examinations 2023 after the approval of the university's statutory bodies.
According to the policy, predefined difficulty levels for questions in all professional exams have been established, with each paper comprising 20% hard questions, 20% easy questions, and 60% moderate questions. The papers will encompass a mix of multiple-choice and short essay questions, evaluating knowledge, application, and problem-solving abilities.
Paper setters will be selected based on their subject matter expertise, a foundational understanding of medical education (e.g., certification in health profession education), and prior experience in evaluating medical students. The assessment process will be conducted consistently, adhering to a predefined format to maintain uniformity in question styles, difficulty levels, and grading criteria. As per the new policy, easy questions may be positioned at the beginning of the assessment to boost examinee confidence, while challenging items must be placed toward the end to differentiate candidates' competencies.
The policy further ensures that paper setters align their question creation process with Bloom's Taxonomy of Cognition to ensure a comprehensive assessment of students' understanding and capabilities. They will use resource books recommended in the curriculum to draw questions, ensuring alignment with prescribed content.
The Examiners Toolkit outlines that after a paper is finalized by the initial paper setter, it will undergo an internal review by another subject expert to assess its structure, difficulty level, alignment with learning objectives, and assessed domains. The feedback will be shared with the paper setter to facilitate revisions and improvements, enhancing question quality. Once the feedback is incorporated and revisions are made, the final version of the question paper will be made available for examination.
Questions selected after undergoing a review process will be stored in the Question Bank using software that can categorize items based on discipline, organ system/clinical presentation, question type (e.g., MCQ, SEQ, OSPE/OSCE), years of first and subsequent uses, difficulty index, and discrimination index on first and subsequent uses.
The cybersecurity of the Question Bank will be maintained through cloud computing, document encryption with physical keys, strong and regularly changed passwords, two-factor authentication (2FA), and biometric security.
To implement these standard operating procedures (SOPs), 56 committees of head examiners will be constituted for both MBBS and BDS programs, each comprising five senior faculty members, including three from public sector and two from private sector colleges. Three sets of question papers will be prepared for each subject, with only one set randomly selected for each examination. These papers will be prepared using approved resource and reference books, and efforts will be made to avoid repeating questions consecutively in two exams. The standardization protocols decided by the CCE will ensure standards of difficulty levels, cognitive level and integrative approach for basic and clinical subjects.
Chairing the CCE meeting on Saturday, UHS Vice-Chancellor Professor Ahsan Waheed Rathore emphasized the university's goal to align the examination process with international standards. He highlighted the crucial role of paper setters in shaping the learning journey of countless students and acknowledged their contribution as "the architects of knowledge evaluation and assessment construction that inspire learning and growth."
He added that UHS is adopting a comprehensive, strategic, and systematic approach to assessment in line with the learning outcomes of the new modular curriculum introduced earlier this year, using assessment methods that are valid and reliable. Professor Rathore also stated that the guidelines would be periodically reviewed to ensure their ongoing relevance and effectiveness.
The meeting was attended by Pro- Vice-Chancellor Professor Nadia Naseem, Convener of the Committee Dr. Uruj Zehra, Controller of Examinations Dr. Muhammad Abbas, Director Medical Education Dr. Khalid Rahim, and other members.
