Chalk & Wire has effectively given us the ability to output a personal growth assessment (PGA) for each student that compares their score with the average of their fellow peers. Just as the dashboard of a car portrays essential information about a vehicle’s condition, the PGA can enlighten a student on his or her progress towards their learning outcomes.
With a desire to be able to directly measure student progress on learning outcomes, Oral Roberts University (ORU) made a strategic decision to develop a whole person assessment (WPA) model and to implement it through a university-wide eportfolio system. As students naturally develop their proficiencies and capacities through ORU’s learning experiences (courses, activities, spiritual encounters, and so on), we wanted to use an eportfolio tool that could easily capture evidence of student work, be able to assess that work, and then be able to report out the results to students, faculty and internal or external stakeholders.
Traditionally, grades alone have been used to show that students have passed an assortment of courses prescribed in a degree plan sheet. However, we wanted students not only receive a grade point average (GPA) from courses they have completed, but also to receive a professional growth assessment (PGA) on key learning outcomes.
Chalk & Wire has effectively given us the ability to output a PGA for each student that compares their score with the average of their fellow peers. Just as the dashboard of a car portrays essential information about a vehicle’s condition, the PGA can enlighten a student on his or her progress towards their learning outcomes. The PGA is a set of composite scores that reflect direct learning performance on the ORU learning outcomes. The summary figures for each learning outcome are an average of the eportfolio results from the assignments designed to measure different aspects of each outcome. Students can access their PGA scores in the same manner they access grades online.
With input from all levels of the university, including students and the board of regents; the faculty selected the proficiencies/capacities (Table I) that monitor student fulfillment of the learning outcomes identified in ORU’s mission statement.
Table l. ORU Student Learning Outcomes
ORU publicly committed to these specific student learning outcomes in 2004 by making the eportfolio a requirement for degree completion. Each student is required to maintain a personal library of artifacts (class assignments or other exhibits), separated into two portfolios—one for general education and one for their major or program. Each major eportfolio includes appropriate university outcomes, as well as outcomes specific to the area of study. Portfolio submissions are assessed using rubrics that interface online with the artifacts and automatically enter the results into an assessment database. This requirement is emphasized in the ORU catalog, on our web site, and in a variety of communications with both potential and current students.
Every course syllabus indicates the level at which the course contributes to each learning outcome and its associated proficiencies/capacities. We have also developed a complete curriculum map showing how courses contribute to the overall mission by linking every course to the student learning outcomes.
ORU determines and communicates what rubric-evaluated criteria serve as evidence that outcomes have been achieved. Each rubric explicitly informs students in advance of the expected performance criteria. After a faculty member evaluates an artifact using an online rubric, the student immediately receives feedback and knows the extent to which the assignment specific outcomes were achieved.
For example, the general education rubric for critical thinking artifacts includes such criteria as a clearly identified purpose and hypothesis, accurate and appropriate evidence (data), understanding of key concepts, crucial and consistent assumptions, and valid inferences and conclusions supported by content. Each rubric explains the levels of performance for each criterion.
In order for the student results in the eportfolio to provide meaningful data, it is necessary to decide which rubrics and/or individual subsections of rubrics, called criteria, contribute to that outcome. For example, students enrolled in the Principles of Chemistry general education lab course write a 250- to 300-word abstract for a meat analysis lab exercise. They are asked to hypothesize and identify one of four types of meat as the healthiest and most cost-effective before they analyze the results of the lab experiment.
Table 2. Complete Rubric for Meat Analysis Lab Abstract
(CHE 101L/Principles of Chemistry Laboratory; 2A-Intellectually Alert-Critical Thinking)
The meat analysis abstract is submitted to faculty members via the eportfolio under the intellectually alert outcome and critical thinking proficiency, where it is assessed according to the critical thinking rubric criteria. Individual students receive assessment feedback for each component from a faculty member via their eportfolio according to the following five-point (zero to 4.0) scale: Exemplary (4.0), competent (3.0), acceptable (2.0), unacceptable (1.0), or not attempted (0.0). The faculty member also has the opportunity to add comments to the assessment to augment the quantitative data. The actual rubric appears in Table 2.
The eportfolio administrator chooses the appropriate items from a list of all rubrics and criteria, so that the selection process results in a truly custom-built standard representing the learning outcome. For example, an academic career score for the learning outcome intellectually alert is calculated by averaging scores from nine general education and several major rubrics that reflect the five proficiencies composing the outcome (see Table 1). A sample student’s view of the assessment results is shown in Table 3. There is a feature in Banner/VISION that shows each student’s eportfolio average composite scores on ORU’s student learning outcomes, displayed in such a way that the student can compare his or her composite scores on each proficiency and capacity composing each of the outcomes with those of all other students.
Table 3. Example of Whole Person Assessment (WPA) Scores
Oral Roberts University has worked towards building an infrastructure that enables the maintenance of direct measurements of student learning outcomes on a basis consistent with conventions of validity and reliability. To do this we have relied heavily on the Chalk & Wire systems to accurately collect students work, provision its assessment and make sense of the data. This project has earned us the 2007 CHEA Award for Institutional Progress in Student Learning Outcomes.
Aside from just being able to generate student PGA scores we are now also using Chalk & Wire for our whole person scholarships (WPS). The WPS program enables students to renew their scholarships based on their assessment scores rather than on grades, athletic performance, or need; since the university already had financial aid packages based on those dimensions.
The program has helped boost enrollment in a time when many or most private institutions have declining enrollments. This was accomplished by actively recruiting and rewarding students with great potential: those who are socially adept, intellectually alert, spiritually alive, physically disciplined, and professionally competent, but are not necessarily honor students or athletes. The institution’s governing board has become engaged in supporting and funding the scholarships, and the program has caught the attention and support of external donors.
Dr. Cal Easterling
Oral Roberts University Institutional Research