What research exists comparing the clinical competence of graduates?

I’m really struggling to find good research that directly compares the clinical competence of graduates from different types of healthcare training programs. Specifically, I’m interested in comparisons across professions, not just within one (like comparing MDs from different medical schools).

For example, is there any robust research that looks at the actual clinical performance of newly graduated Physician Assistants versus Nurse Practitioners versus Family Medicine Residents in a primary care setting? Or maybe research that compares the clinical reasoning skills of new graduate Physical Therapists versus Occupational Therapists?

I’m interested in objective measures of competence, if possible. Things like diagnostic accuracy, adherence to evidence-based guidelines, patient outcomes, or even standardized patient assessments. Self-reported competence is interesting, but I’m really looking for evidence-based performance data.

I’m aware there are differences in curriculum and scope of practice, but I want to see if there are studies that try to control for these differences and isolate the impact of different training approaches on actual clinical performance. Has anyone come across anything like that? I’m open to research in any healthcare field, but my primary interest is in direct patient care.

Answer

Research comparing the clinical competence of graduates exists across various healthcare professions, including medicine, nursing, pharmacy, and allied health fields. The studies often employ diverse methodologies, assessment tools, and outcome measures, reflecting the complexity of defining and evaluating clinical competence. Here’s a breakdown of research areas and findings:

1. Medical Graduates:

• Standardized Examinations: National licensing exams like the USMLE (United States Medical Licensing Examination) in the US, the MCCQE (Medical Council of Canada Qualifying Examination) in Canada, and similar exams in other countries serve as a baseline assessment of knowledge and clinical reasoning skills. While these exams are standardized, some research investigates the predictive validity of these scores in relation to actual clinical performance during residency and beyond. Some studies correlate exam scores with residency performance evaluations or board certification outcomes.
• Residency Performance: Residency programs are crucial training grounds, and research often focuses on evaluating residents’ clinical competence through direct observation, faculty evaluations, patient feedback, and standardized assessments like mini-Clinical Evaluation Exercises (mini-CEX), Direct Observation of Procedural Skills (DOPS), and 360-degree evaluations. Studies may compare the performance of graduates from different medical schools or different training programs within the same specialty, looking at areas like patient management, communication skills, professionalism, and procedural competence. Research also explores the relationship between pre-clinical training (e.g., curriculum design, simulation experiences) and residency performance.
• Objective Structured Clinical Examinations (OSCEs): OSCEs are widely used to assess clinical skills in a standardized and objective manner. Research comparing graduates may involve administering OSCEs at different points in their training (e.g., at graduation, during residency) to track their progress and identify areas for improvement. Studies can compare the OSCE performance of graduates from different medical schools or different training programs. The psychometric properties of OSCEs and their correlation with real-world clinical performance are also areas of investigation.
• Patient Outcomes: Some studies attempt to link physician competence to patient outcomes, such as mortality rates, complication rates, hospital readmission rates, and patient satisfaction scores. This is a complex area of research, as many factors beyond physician competence influence patient outcomes. However, researchers may use statistical methods to adjust for confounding variables and isolate the impact of physician performance. These studies can compare the outcomes of patients treated by graduates from different training backgrounds.
• Simulation-Based Assessments: Medical simulation is increasingly used to assess clinical skills in a safe and controlled environment. Research compares graduates’ performance on simulated tasks, such as managing a medical emergency or performing a surgical procedure. These assessments can provide valuable insights into graduates’ readiness for clinical practice.

2. Nursing Graduates:

• NCLEX-RN Examination: The NCLEX-RN (National Council Licensure Examination for Registered Nurses) is a standardized exam required for licensure as a registered nurse in the United States and Canada. Research examines pass rates on the NCLEX-RN and investigates factors that predict success on the exam, such as pre-licensure nursing program characteristics, student demographics, and academic performance.
• Clinical Performance Evaluations: Nursing programs and healthcare organizations use clinical performance evaluations to assess nursing graduates’ competence in areas such as patient assessment, medication administration, communication, teamwork, and critical thinking. Research compares the clinical performance of graduates from different nursing programs, using evaluation data collected by faculty, preceptors, and supervisors.
• Simulation-Based Education and Assessment: Simulation is widely used in nursing education to provide students with opportunities to practice clinical skills in a safe and realistic environment. Research compares the clinical competence of graduates who have participated in simulation-based education with those who have not, or compares the effectiveness of different simulation modalities. Simulation-based assessments are also used to evaluate graduates’ clinical skills in a standardized manner.
• Transition to Practice Programs: Many healthcare organizations offer transition to practice programs for new graduate nurses to provide them with additional support and training during their first year of practice. Research evaluates the effectiveness of these programs in improving new graduate nurses’ clinical competence, confidence, and job satisfaction. Studies compare the performance of new graduate nurses who participate in transition to practice programs with those who do not.
• Patient Safety Indicators: Researchers may examine patient safety indicators, such as medication errors, falls, and hospital-acquired infections, to assess the impact of nursing graduates’ competence on patient outcomes. These studies can be challenging to conduct due to the many factors that influence patient safety, but they can provide valuable insights into the importance of ensuring that nursing graduates are adequately prepared for clinical practice.

3. Pharmacy Graduates:

• NAPLEX Examination: The NAPLEX (North American Pharmacist Licensure Examination) is a standardized exam required for licensure as a pharmacist in the United States and Canada. Research examines pass rates on the NAPLEX and investigates factors that predict success on the exam, such as pharmacy school curriculum, student demographics, and experiential education experiences.
• Clinical Rotations and Internships: Pharmacy students complete clinical rotations and internships as part of their training, which provide them with opportunities to apply their knowledge and skills in real-world practice settings. Research evaluates the clinical competence of pharmacy graduates during these rotations and internships, using assessments such as direct observation, case presentations, and medication therapy management plans. Studies compare the performance of graduates from different pharmacy schools or different types of experiential education sites.
• Residency Programs: Many pharmacy graduates pursue residency programs to further develop their clinical skills and specialize in a particular area of practice. Research evaluates the clinical competence of pharmacy residents, using assessments such as project presentations, patient care activities, and evaluations from preceptors. Studies compare the performance of residents from different residency programs or different areas of specialization.
• Objective Structured Clinical Examinations (OSCEs): As in medicine and nursing, OSCEs are used in pharmacy education to assess clinical skills such as patient counseling, medication dispensing, and drug information retrieval. Research compares the OSCE performance of pharmacy graduates from different schools or different training programs.
• Patient Outcomes: Some studies attempt to link pharmacist competence to patient outcomes, such as medication adherence, blood pressure control, and cholesterol management. This is a complex area of research, but studies may use statistical methods to adjust for confounding variables and isolate the impact of pharmacist performance. These studies can compare the outcomes of patients managed by pharmacists from different training backgrounds.

4. Allied Health Graduates:

• Profession-Specific Licensure Examinations: Allied health professions, such as physical therapy, occupational therapy, and speech-language pathology, typically require graduates to pass a profession-specific licensure examination to practice. Research examines pass rates on these exams and investigates factors that predict success.
• Clinical Placements and Internships: Allied health students complete clinical placements and internships as part of their training. Research evaluates the clinical competence of graduates during these placements, using assessments such as direct observation, case presentations, and patient documentation. Studies compare the performance of graduates from different educational programs or different types of clinical sites.
• Competency-Based Education: Many allied health programs are adopting competency-based education models, which focus on ensuring that graduates have the specific knowledge, skills, and attitudes required for entry-level practice. Research evaluates the effectiveness of these models in improving graduates’ clinical competence.
• Patient Outcomes: Some studies attempt to link allied health professional competence to patient outcomes, such as functional status, pain levels, and quality of life.

Common Themes and Challenges:

• Defining Competence: A major challenge is defining and operationalizing "clinical competence." It’s a multifaceted construct encompassing knowledge, skills, attitudes, and behaviors. Researchers use various frameworks and models to guide their assessments, but there’s no single universally accepted definition.
• Assessment Tools: The choice of assessment tool influences the results. Standardized exams, OSCEs, direct observation, simulations, and patient outcomes all provide different perspectives on competence. Each tool has its strengths and limitations, and researchers must carefully consider the validity and reliability of their chosen methods.
• Context Matters: Clinical competence is context-dependent. Performance can vary depending on the patient population, the practice setting, and the available resources. Studies need to account for these contextual factors when comparing graduates.
• Longitudinal Data: Assessing long-term clinical competence requires longitudinal data, tracking graduates’ performance over time. These types of studies are more resource-intensive but provide valuable insights into the trajectory of professional development.
• Confounding Variables: Many factors can influence clinical competence, including pre-existing knowledge and skills, personality traits, motivation, and the quality of the learning environment. Researchers need to use appropriate statistical methods to control for these confounding variables when comparing graduates.
• Focus on Improvement: Research in this area should not only compare graduates but also identify areas for improvement in educational programs and training practices. The goal is to enhance the quality of healthcare professionals and improve patient outcomes.
• Use of Technology: Technology such as telehealth, remote patient monitoring, and artificial intelligence is changing healthcare, and research is exploring how these technologies impact the clinical competence of graduates.

In summary, a substantial body of research exists that compares the clinical competence of graduates across various healthcare professions. The research employs a variety of methodologies and assessment tools, and it often focuses on identifying factors that predict success in clinical practice and areas for improvement in educational programs. However, defining and assessing clinical competence remains a complex and ongoing challenge.