Standard 11, Outcomes Assessment Outcomes of the DVM program must be measured, analyzed, and considered to improve the program. Student achievement during the pre-clinical and clinical curriculum and after graduation must be included in outcome assessment. New graduates must have the basic scientific knowledge, skills, and values to provide entry-level health care, independently, at the time of graduation. The school/college must develop relevant measures and provide evidence that students/graduates have attained the following competencies: 1. 2. 3. 4. 5. 6. 7. 8. 9. comprehensive patient diagnosis (problem solving skills), appropriate use of clinical laboratory testing, and record management comprehensive treatment planning including patient referral when indicated anesthesia and pain management, patient welfare basic surgery skills, experience, and case management basic medicine skills, experience and case management emergency and intensive care case management health promotion, disease prevention/biosecurity, zoonosis, and food safety client communications and ethical conduct critical analysis of new information and research findings relevant to veterinary medicine Colleges must have annual and cumulative NAVLE passing percentages that statistically fall within the range of the upper 95% of all accredited colleges. Colleges must evaluate five-year trends in student attrition. Significant evidence of a rising trend in the annual proportions of attrition must be explained and corrective remediation measures implemented and described. Colleges must evaluate five-year trends in one-year post-graduation proportionate employment. Significant evidence of a declining trend in the annual proportions of graduates achieving employment within one year of graduation must be explained and corrective remediation measures implemented and described. The following information is provided to explain why changes in use of the NAVLE pass rate are being proposed and the underlying methodology: A nonparametric method of examining the distribution of scores is advisable because the current NAVLE pass rates are not normally distributed among accredited colleges (Shapiro-Wilk P-value < 0.00001) There are two ways to construct an interval that captures 95% of the data from the 2010-2011 sample Based on the 2.5th and 97.5th percentiles Based on the 5th and 100th percentiles The latter was chosen because there are colleges with 100% passing rates. A college with passing rate of 77% (77 out of 100): 95% confidence interval = 67.5% - 84.8% 2010-2011 NAVLE scores for accredited colleges 95% confidence interval is 88% - 100% This college would not have an acceptable passing rate A college with passing rate of 80% (80 out of 100): 95% confidence interval = 70.8% - 87.3% 2010-2011 NAVLE scores for accredited colleges 95% confidence interval is 88% - 100% This college would just barely miss having an acceptable passing rate Advantages of this approach Uses actual accredited colleges’ data to establish a “reference range” for comparison with candidate colleges’ NAVLE passing rates Does not require arbitrary cutoffs (e.g., 80%) Does not make assumptions about underlying probability distributions of accredited colleges’ NAVLE scores Uses defensible and replicable exact Clopper-Pearson binomial confidence intervals 95% confidence intervals - while arbitrary - are widely used and accepted in statistical analyses If the proposed changes to Standard 11are finalized in current form, the following revisions to the self-study requirements listed in the COE manual, Section 12.11 will be revised as follows (additions are underscored, deletions struck). Data to demonstrate outcomes of the educational and institutional program(s) may be collected by a number of means including, but not limited to, surveys, interviews, focus groups, self-assessments, third-party provider, information held by the college, and other. Where appropriate, the data must be analyzed/summarized for brevity. Except for North American Veterinary Licensing Examination (NAVLE), the Council does not assign numerical values to document levels of achievement for students in any of the outcome delineators, but closely analyzes trends for the college. Trends that imply significant decrease(s) in student achievement over a five-year period may imply deficiencies in the program. The trends are used by the Council in its analysis of the compliance of the college with the Standards. In the case of declining trends in the delineators, the college must provide an explanation for the decline(s), and must provide a plan to reverse the trend(s). 12.11.1. Student educational outcomes must include, but are not limited to: 12.11.1.a. NAVLE school score report data and passage rates over the past five years (Table A), Each college must submit a copy of the annual NAVLE School Score Report with the AVMA-COE Interim Report each year. The Council on Education expects that 80% or more of each college's graduating senior students sitting for the NAVLE will have passed at the time of graduation. Colleges with recurring passing percentages less than 80% the upper bound of an exact binomial 95% confidence interval of the annual passing percentage will exceed the 5 th percentile of NAVLE passing percentages of all accredited colleges each year. If the confidence interval of the cumulative passing percentage for two successive years fails to include the 5th percentile of all accredited colleges, the college will be placed on Limited Accreditation. Colleges with passing percentages less than 80% If the exact binomial 95% confidence interval of the cumulative passing percentage fails to include the 5th percentile of passing percentages of all accredited colleges for four (4) successive years will, for cause, be placed on Terminal Accreditation. 12.11.1.b. student attrition rates with reasons (Table B), Each college must submit data on absolute attrition every year. The Council on Education expects that a significantly increasing (positive) trend in absolute 12.11.2. attrition from the college will be explained, and that the college will implement steps for arresting the trend. Five-year annual proportions in absolute attrition will be tested for positive trend using the exact Cochran-Armitage test; statistical significance is defined as a two-sided probability value 0.05. If the five-year trend remains positive for three successive years after corrective remediation measures have been implemented, the Council will take action that may include, but not be limited to, finding the college not in compliance with Standard 11, Outcomes Assessment. 12.11.1.c. the learning objectives for each of the nine listed competencies, and a summary of the analysis of evidence-based data collected for each of the nine listed competencies used to ensure that graduates are prepared for entry level practice (please note that a listing of core and elective blocks does not constitute evidence of learning). Evidence of student learning outcomes for clinical competencies must be obtained by direct measures. These may include capstone experiences, student portfolios, standardized clinical proficiency exams, or other evaluations of clinical performance based on measurable and published program objectives. Indirect measures should not be used as the sole determinants of clinical competency outcomes. Examples include employer surveys and student course or rotation grades 12.11.1.d. employment rates of graduates (within one year of graduation) (Table C), Each college must submit data on one-year post –graduate employment every year. The Council on Education expects that a significantly declining (negative) trend in proportionate employment from the college will be explained, and that the college will implement steps for arresting the trend. Five-year annual proportionate employment will be tested for positive trend using the exact Cochran-Armitage test; statistical significance is defined as a two-sided probability value 0.05. If the five-year trends remain negative for three successive years, the Council will take action that may include, but not be limited to, find the college to be not in compliance with Standard 11, Outcomes Assessment. 12.11.1.e. assessments of graduating seniors; and assessments of alumni at some postgraduation point (for example, three and/or five years post-graduation) assessing educational preparedness and employment satisfaction, 12.11.1.f. assessments of employers of graduates to determine satisfaction with the graduates, 12.11.1.g. assessments of faculty (and other instructors, for example interns and residents) related to such subjects as adequacy of clinical resources, facilities and equipment, library and information resources, etc.; and preparedness of students entering phases of education, and 12.11.1.h. additional assessment that might assist the college in benchmarking its educational program. Institutional outcomes. 12.11.2.a. Describe how the college evaluates progress in meeting its mission (for example, benchmarking with other institutions, etc.). 12.11.2.b. Describe the adequacy of resources and organizational structure to meet the educational purposes (dean should provide). 12.11.2.c. Describe outcomes assessed for college activities that are meaningful for the overall educational process (for example, scholarly activity of the faculty, faculty awards, faculty and staff perception of teaching resources, student satisfaction with the educational program, teaching improvement benchmarks, and others). If your program assesses other outcomes, briefly describe the results. 12.11.2.d. Describe how outcomes findings are used by the college to improve the educational program (give examples) Table A – NAVLE Year Students taking exam(s) Students passing exam(s) Average scores Table B – Attrition Entering Class Attrition* Reason for Relative Attrition Academic Failure/Additional Personal Program Absolute Attrition** Number Percentage * Students that are either withdrawing from the program or moving to a different (earlier) class ** Students who leave and never return Table C – Employment Rates Graduating Class # Graduates Who Received Employment or Advanced Training Offers / # Completing This Question, and (%) Mean # Employment or Advanced Training Offers Received

© Copyright 2018