1	Welcome to Assessment 101
2	Assessment 101, Session #1 Goals for this session Understand assessment terminology Understand the different types of goals and student outcomes Consider methods of obtaining feedback about accomplishment of student learning goals Become familiar with various ways of using assessment information
3	One definition of assessment: “A process that asks important questions about student learning, gathers some meaningful information on these questions, and uses the information for academic improvement.” Source: Sinclair Community College, as referred to by Denney, 1996 in Palamba and Danta’s Assessment Essentials. 1999.
4	Another definition of assessment: “A rich conversation about student learning informed by data.” Source: Ted Marchese, American Association for Higher Education
5	Assessment can help us answer: How well do student outcomes match the goals we have established? What experiences will students have as a result of completing their general education requirements? Is what we say we want students to learn (in our mission statement) what they actually learn (curriculum)?
6	Assessment can help us answer: How is what Cumberland students learn unique? How can we demonstrate it? What value does University of the Cumberlands add to students’ acquisition of knowledge, skills, and attitudes? What impact does the University have on changes in students during their four years here? How do we know?
7	Components of a strong academic program
8	Program Planning Process
9	Student Learning Outcomes Content: Knowledge of a subject matter (Cognitive Learning) Skill Acquisition: Comprehension of a topic, demonstration of a competency, etc. (Behavioral Learning) Attitudes: Awareness, interest, concern, etc. (Affective Learning) *Outcomes may be a combination of content knowledge, skills, and attitudes.
10	Content/Cognitive Learning Examples of Direct Measures: Subject tests (ETS field tests, Academic Profile, etc.) Portfolios Coursework in Capstone Courses/Senior Projects
11	Content/Cognitive Learning Examples of Indirect Measures: Feedback from internship, co-op, or student teaching supervisors Students’ self reports of knowledge gained
12	Skill Acquisition/Behavioral Learning Examples of Direct Measures: Juried performance (music, speech, art exhibit, poster session, etc.) Portfolios Assignments and Coursework in Capstone Courses
13	Skill Acquisition/Behavioral Learning Examples of Direct Measures: Major Project (research project, major paper, case study, group project) Mathematical reasoning tests, foreign language competency tests, critical thinking tests, etc.
14	Skill Acquisition/Behavioral Learning Examples of Indirect Measures: Feedback from internship, co-op, or student teaching supervisors Students’ self reports of skill attainment
15	Attitudes/Affective Learning Examples of Direct Measures: Survey of students’ responses to value-laden issues Test of students’ recognition of and understanding of ethical issues
16	Attitudes/Affective Learning Examples of Indirect Measures: Feedback from employers, internship supervisors, etc. Pre and post tests or surveys that measure changes in attitudes, values, or beliefs Journals, interviews, focus groups Surveys of graduates
17	Indirect Measures of Student Learning Alumni, employer and student surveys Exit interviews of seniors Follow-up studies of graduates
18	Non-Measures of Student Learning Students, employers, and graduates’ satisfaction ratings Number of courses students have completed Grades/G.P.A. Graduate/professional school acceptance rates
19	The Assessment Process First: Develop goals that address the student outcomes that you want to achieve. Then: Design measures that assess those goals. Every goal is measurable.
20	Measurement “I prefer to have soft or rough measures of important goals rather than highly precise, quantitative measures of goals that no one much cares about.” Source: Patton, Michale. 1997. Utilization-Focused Evaluation: The New Century Text, Thousand Oaks, CA: Sage Publishers 3^rd Edition.
21	Principles of Measurement Quantitative vs. qualitative measures Student evaluations vs. outcome assessment Why are grades not a good indicator of student learning? The value-added approach to measuring outcomes.
22	Measurement Criteria: Performance Targets May be based on the previous year’s performance (it’s ok to develop baseline data) May be based on norms/benchmarks from other comparable programs (national standards) May be based on research from experts in the field (professional associations, accrediting agencies, etc.)
23	Measurement Criteria: Performance Targets Should be monitored periodically Should be updated when appropriate
24	Measurement Criteria Before collecting data, consider what level of knowledge, skill, or behavior change, etc. is excellent, average, poor. Decide, before you conduct the measurement, what the results will mean. What do the results tell you about achievement of student learning outcomes?
25	Measurement Criteria What are the “cutoff points” that delineate various levels of goal attainment? What level is problematic, requiring additional action?
26	Measurement Criteria For example: What if 20% of students indicate that their learning experiences at Cumberland did not sufficiently prepare them to use technology in their careers/at graduate school? Is this an acceptable level? If this is not an acceptable level, what action should be taken in response
27	Measurement Criteria For example: What if 20% of students indicate that their learning experiences at Cumberland did not sufficiently prepare them to use technology in their careers/at graduate school? What if students indicate that they need additional computer skills and more refined analytical skills? If both are issues that need to be addressed, which should take priority? Faculty can answer these questions more easily by defining the performance criteria before administering the assessment.
28	The Importance of Using Multiple Measures A single instrument can not capture all of the elements of student learning. Goals are not one-dimensional; most programs have multiple goals or multi-faceted goals for student learning Patterns of evidence demonstrate student learning. A single measure may be inadequate evidence on which to base accurate conclusions.
29	Characteristics of a good Assessment Plan Ongoing Manageable (Do not try to do everything at once; assessment can be phased in over a period of time) Faculty use results for decision making Student understand their role in assessment
30	How can assessment results be used in your department to improve your program? Curriculum in courses Pedagogy in courses Structure of major, requirements, etc. Advising
31	How can assessment results be used in your department to improve your program? Best use of resources Support for requests for additional resources Staffing decisions
32	Assessment 101 Session #2 Goals for this session: Be aware of some of the national trends in assessment. Be familiar with research that tells us about assessment and student learning. Become familiar with other institutions’ assessment plans and procedures.
33	Southern Association of Colleges and Schools Expectations University-wide and departmental assessment plans should provide evidence of accomplishment of goals and purposes. General education requirements should be consistent with the institution’s mission and goals. Objectives for general education should be explicitly stated and widely publicized.
34	Southern Association of Colleges and Schools Expectations Results of the assessment of student learning should be linked to planning and budgeting. Assessment results should influence organizational change.. Faculty clearly understand assessment philosophies and methods and they should participate in university-wide and program level assessment.
35	Southern Association of Colleges and Schools Expectations Bottom line: SACS wants departments to have a method for ensuring and improving educational quality
36	"“In a 1995 Campus..." “In a 1995 Campus Trends study, 76% of responding institutions indicated they had made curriculum or program changes based on assessment findings.” Source: Et-Khawas, Elaine. Higher Education Panel Report, No. 85, Washington, D.C., American Council on Education.
37	"“Because assessment brings such..." “Because assessment brings such important information to the table, the number of (curriculum) changes that are made in the absence of assessment results should be small.” Source: Palomba, Catherine and Trucy Banta. 1999. Assessment Essentials. San Francisco: Jossey-Bass Publishers.
38	Trends in Assessment Most institutions’ student learning goals are similar ( for example, communication skills, critical thinking skills, ability to work in diverse settings, etc.) but assessment of those goals varies considerable.
39	Trends in Assessment Assessment is integrated into curriculum Assessment is peripheral to development, planning, and budgeting Assessment is conducted across the university Assessment is conducted for curriculum *Results are used to improve students’ learning
40	Assessment of General Education General education exam (i.e., Academic Profile) Writing proficiency exams Samples of students’ work (speeches, performances, exhibits, etc.) Current student/alumni/employer surveys Critical thinking measures
41	Assessment of the Major Virginia Tech http://fbox.vt.edu:10021/admin/assess/deptrep.html Montana State University http://www.montana.edu/aircj/assess/majors/majors9899/ University of Central Arkansas http://www.uca.edu/assess/asomdex.htm
42	Assessment Web Sites Alverno College www.alverno.edu Truman State University www2.truman.edu/assessment98/ Southeast Missouri State University www2.semo.edu/provost/aspnpg.htm Ball State University www.bsu.edu/provost/IRAA/index.html St. Olaf College www.stolaf.edu
43	Characteristics of “strong” Assessment Programs Faculty support using assessment data to improve student learning Assessment is “built in” to the curriculum. Assessment and planning are closely intertwined. Students understand assessment and take ownership and interest in their own learning.
44	Challenges to developing a Comprehensive Assessment Plan Increasing faculty involvement in assessment. Managing the workload implications of conducting assessment. Motivating students to do their best on assessment activities. Using assessment results to improve programs. Identifying methods to realistically measure outcomes.
45	Examples of what assessment tells us about student learning “The most important influence on the development of critical thinking abilities is providing students with structured opportunities to integrate information across courses and across in and out of class activities.” Source: Pike, Gary. “Using Surveys to Measure Critical Thinking Outcomes.” Assessment Update. May-June 1997, Vol. 9, No. 3.
46	Examples of what assessment tells us about student learning “One…conclusion we can make is that the impact of college is largely determined by the individual’s quality of effort and level of involvement in both academic and non-academic activities.” Source: Pascarella, Ernest and Patrick Terenzini. 1991. How College Affects Students: Findings and Insights from Twenty Years of Research. San Francisco: Jossey-Bass Publishers.
47	Examples of what assessment tells us about student learning “Anxiety, fear of failure, and low self-esteem are associated with surface approaches to learning.” Entwhistle “urges university faculty to build up students’ sense of control over their work and get them to exercise responsibility for their own learning.” Source: Marchese, Theodore. 1997. “The New Conversation about Learning.” Assessing Impact. Washington, DC: Ammerican Association for Higher Education.
48	Examples of what assessment tells us about student learning “Paul Ramsden has been able to identify characteristics of courses in which students take a surface approach to learning: the courses obsess over coverage; the huge amount of material they cram in precludes opportunity to pursue topics in depth, students have little choice about what and how to study; and such courses have a threatening. Anxiety-riddled instructional and examining environment.” Source: Marchese, Theodore. 1997. “The New Conversation about Learning.” Assessing Impact. Washington, DC: Ammerican Association for Higher Education.
49	Examples of what assessment tells us about student learning “The more a teacher can emphasize: learner choice and feedback intrinsic motivators and natural curiosity rich, timely, usable feedback, coupled with occasions for reflection
50	Examples of what assessment tells us about student learning “The more a teacher can emphasize: Active involvement in real-world tasks, Emphasizing higher-order abilities done with other people in high-challenge, low-threat environments that provide for practice and reinforcement…the greater the chances he or she will realize the deep learning that makes differences in student lives Source: Marchese, Theodore. 1997. “The New Conversation about Learning.” Assessing Impact. Washington, DC: Ammerican Association for Higher Education.
51	Examples of what assessment tells us about student learning “During their first year, however, they make gains to approximately the same degree as their peers. However, their college experiences including courses taken, hours spent studying, perceptions of faculty, and experiences with racial/gender discrimination, are very different from their more traditional peers.” “Students’ out-of-class and in-class experiences are interrelated; both affect the student’s learning and development Source: NCTLA. http://cac.psu.edu/nctla/projects.html
52	Examples of what assessment tells us about student learning Learning communities involve student taking two or more courses (usually general education courses) that link disciplines and explore a common theme. They may involve students living together in communities in residence halls. Learning communities have been implemented at Evergreen State College, SUNY Stony Brook, University of Maryland-College Park, and Ball State University.
53	Examples of what assessment tells us about student learning “Learning communities enhance communication between students and faculty, help students build supportive relationships with their peers, and result in greater perceived gains with respect to intellectual development.” Source: Tinto, Vincent. “Classrooms as Communities: Exploring the Education Character of Student Persistence.” Journal of Higher Education. 1997, 68(6), 599-623.
54	Examples of what assessment tells us about student learning Research on the impact of additional learning strategies, such as service learning, first year experience courses, study abroad, team-taught courses, etc. are based on assessment results and have implications for curriculum design.
55	Other issues, questions, or topics for discussion? Next steps?