CTLA Assessment Plans and Reports
2013-2014
CTLA assessed three outcomes in 2013-2014. We assessed Goal 2 (Identify and provide resources for faculty to improve teaching) Outcome #1: Faculty will use resources provided by CTLA to improve their teaching. We selected to assess the CTLA Resource Faculty Learning Communities, facilitated by CTLA and supported by CTLA funds for stipends and resource materials. We assessed the components (1) can identify an application to teaching, (2) has implemented the improvement, and (3) has collected indications of improvement. We used a survey, sent to members of the two most recent Faculty Learning Communities that were completed, to self-report their results. Our respondents met all three benchmarks of 75% for both identifying and applying a change to improve their teaching and 25% for having collected indications of improvement. We are finished measuring this outcome at this time.
CTLA also assessed Goal #3 (Orient faculty to career development processes and milestones and resources) Outcome #1 New faculty will be able to identify campus sources for important information regarding their roles and responsibilities, and Outcome #2 New faculty will feel a sense of connection to the campus community. We assessed Outcome #1 components (1) will be able to identify campus sources of important information regarding their roles and responsibilities and (2) will indicate that faculty orientation was helpful or very helpful in their transition to their faculty role on campus. We surveyed faculty at the end of their year-long orientation program, instead of in the following year, so that we could also assess those in one-year appointments. Our benchmark of 80% was not met. We assessed Outcome #2 components (1) will indicate that they interacted with a campus community member outside their own department or school that was an orientation guest speaker, and (2) will indicate that they stayed in contact with a fellow new faculty member outside their own department or school. This outcome was partially met because, although it was not met for those assessed a year later, it was met for those assessed the same year. Because we were able to double the number of assessment respondents when assessing the outcome in the same year instead of a year later, we had more indicators and more confidence in results. We are finished measuring this outcome at this time. We believe we need to realign the goals for the program with the assessment outcomes and reassess after that process.
Link to Full Report
Link to 2012-2015 3-year Assessment Plan
2012-2013
CTLA assessed two outcomes in 2012-2013. We assessed Goal 1 (Provide resources for faculty and staff to integrate technology in support of students learning and faculty/staff productivity) Outcome #1: Faculty and staff will be able to implement new technology. We looked for the components that they had basic knowledge, could identify potential applications and had implemented the technology. Using a survey, we provided them with a list of technology training that they had completed which was applicable to the Online Instruction Developers Certificate program. We asked them how they had applied any of that training. Our respondents surpassed our benchmark of 30% because 70% could provide an example of how they implemented one of the technologies in their teaching. We are finished measuring this outcome at this time.
CTLA also assessed Goal #3 (Orient faculty to career development processes and milestones and resources) Outcome #1 New faculty will be able to identify campus sources for important information regarding their roles and responsibilities. Both our pool and respondent numbers were small. We surpassed the 80% benchmark that faculty would report orientation as helpful or very helpful with a result of 100%. We did not meet the 80% benchmark for the percentage of faculty who responded with all correct responses to our sample of knowledge questions about campus resources. Our results were 66.7%. With the small number of respondents, we can only reach our benchmark with complete and correct answers from all respondents. We need to rewrite the component and benchmark to be able to distinguish whether we can show difference and improvement over time when we have a small pool and number of respondents, which is typical for this program. We will also change when we conduct our survey to see if we can increase our pool and number of respondents.
Link to Full Report
2011-2012
The Center for Teaching, Learning, and Assessment assessed Goal 1, Outcomes 1-5 for Oncourse technical training. We also assessed Goal 1, Outcomes 2-5 for technical training taught in the fall semester 2011. Here are the outcomes of Goal 1--Provide resources for faculty and staff to integrate technology in support of student learning and faculty/staff productivity.
- Faculty and staff will be able to implement new technology.
- Faculty will report that they use technology to support their students’ learning.
- Faculty will report that they use technology to increase student engagement.
- Faculty will report that they use technology to increase prompt feedback to students.
- Faculty and staff will use technology to enhance their productivity.
For the Oncourse technology training, all five outcomes were achieved at 100% and met our benchmark of 60% of trainees implementing the technology to support Outcomes 1-5. For other technology training, we met our benchmark of 50% of trainees implementing the technology to support Outcomes 2 and 5, supporting student learning and enhancing productivity. Results were under 50% of trainees implementing the technology to support Outcomes 3 and 4, increasing student engagement and increasing prompt feedback to students.
Our assessment of the Oncourse training over time has led us to believe we have a good process in place and can turn our future assessment attention to other outcomes. Our results for technology training may have had lower results for Outcomes 3 and 4 because we included WCMS Faculty and Staff Web Page technology training in our group of courses assessed and respondents may not have even intended to use those for student engagement or prompt feedback to students. In fact, that technology would not typically be suitable for prompt feedback to students due to its public nature. In the next iteration, we would like to assess the technology training classes that we use to support online course development, so it should be more relevant to these outcomes.
We also assessed Goal 3, outcomes 1-2. Here are the outcomes of Goal 3--Orient faculty to career development processes and milestone and resources.
- New faculty will be able to identify campus sources of important information regarding their roles and responsibilities.
- New faculty will feel a sense of connection to the campus community.
Our result for Outcome 1 was that 0% answered all of the knowledge questions correctly although all of them answered 80% correct and 100% of respondents rated the orientation series helpful or very helpful. For Outcome 2, 50% achieved the measure of connection to the campus community. Neither result met our benchmark of 80%. Of the eight faculty sent the assessment survey, two responded. Low participation was a factor that affected our ability to use these results to assess the program. Right now, the benchmark for outcome 1 is written to measure the percentage of respondents who achieve all correct answers for the knowledge questions asked, even if those respondents were not able to attend the session where that knowledge was shared. Although ours attained 80% correct questions, that counted as 0% reaching 100%. We may gain more helpful assessment information by establishing a benchmark for the % of answers correct. We need to continue to assess this orientation program and look for a way to increase our response rate.
Link to Full Assessment Reports
2010-2011
The Center for Teaching, Learning, and Assessment (CTLA) assessed the outcomes
- Faculty and staff will be able to implement new technology.
- Faculty will report that they use technology to support their students’ learning.
- Faculty will report that they use technology to increase student engagement.
- Faculty will report that they use technology to increase prompt feedback to students.
- Faculty and staff use technology to enhance their productivity.
- Faculty will use resources provided by CTLA to improve their teaching.
Faculty assessed for ability to implement new technology after Oncourse training were measured in their small groups and surpassed the benchmark of 80% that they could complete the tasks on their own as a small group.
The respondent rate for the other outcomes surveyed in psring semester after fall technology training was very low, with 4 respondents to one survey and 7 respondents to another survey. We need to make changes in how we assess this so that we can gather more data. We anticipate trying a different survey instrument with features that can send reminders to those sent the anonymous survey who have not yet completed a survey.
The outcome that faculty will use resources provided by CTLA to improve their teaching was assessed for the resource of a Faculty Learning Community experience. Although the average ratings for this year’s Community had a small decrease when compared to previous the Faculty Learning Community ratings, all of the results this year indicated higher than medium impact for all elements rated. Having now collected outcomes of two Faculty Learning Communities, a general benchmark of 7.0 can be adopted for ratings. We did not achieve that this year in all ratings. The rating for understanding of and interest in the scholarship of teaching is lower than we would like, although close. This assessment indicates that it would be worthwhile to include more about the scholarship of teaching in future Faculty Learning Community experiences. Both years of ratings came in under 7 for this element. Other opportunities to increase knowledge, skills, and reward for scholarship of teaching and learning could also affect the ability to increase impact when in a Faculty Learning Community.
We intended to assess the outcome "New faculty will be able to identify campus sources of important information regarding their roles and responsibilities." Due to the low number of new faculty in continuing positions attending the majority of sessions in the orientation series, confidentiality issues caused us to postpone this assessment to another year.
Link to Full Assessment Reports
2009-2010
The Center for Teaching Learning, and Assessment assessed two goals. Goal One was Technology Training and Resources for the outcome that faculty and staff will be able to implement new technology. Goal 2 was Resources for improving teaching and learning for the outcome that faculty will use resources provided by CTLA to improve their teaching.
For goal 1, We assessed Oncourse For Your Course training. We checked whether training attendees could do tasks they were taught on their own, with help, or not accomplish the task. We also assessed whether they applied the training by implementing the new technology they learned in their courses.
Eighty-eight percent of workshop trainees were able to accomplish the assessment tasks on their own, surpassing the benchmark of 80%. Of the eight faculty who attended the Oncourse for Your Course training, three responded to our survey asking which features of Oncourse they used in their courses. All three used Oncourse in their courses.
For Goal Two, we tracked the number of faculty and staff who used our CTLA Lab resource, both in training classes and as individual users not in classes. This lab area can be particularly useful to adjunct faculty if they need a quiet place to work on their grading or other aspects of their courses. Fifty-three individual uses and 33 training class uses occurred in the year 2009-2010. The highest individual use tracked was using Oncourse in our Lab. Because Oncourse is available on any campus or home computer that has Internet access, this suggests to us that individuals who work on Oncourse in our lab are choosing the lab environment as a resource of choice.
Link to Full Assessment Reports
2008-2009
The Center for Teaching, Learning, and Assessment (CTLA) assessed the outcomes
- Faculty and staff will be able to implement new technology.
- Faculty will report that they use technology to support their students’ learning.
- Faculty will report that they use technology to increase student engagement.
- Faculty will report that they use technology to increase prompt feedback to students.Faculty will use resources provided by CTLA to improve their teaching.
CTLA used embedded application tasks during training to assess faculty and staff ability to implement new technology by requesting that participants complete tasks. 89% of participants could complete tasks on their own when assessed in Oncourse for Your Course training. 81% of participants did participate and complete, on their own, a variety of tasks assessing technology use skills learned in other technology training courses.
To assess outcomes that faculty are using technology to support students’ learning, increase student engagement, and increase prompt feedback to students, CTLA assessed how many faculty who had participated in the Team-Based Learning (TBL) workshop and/or Team-Based Learning Faculty Learning Community, had implemented the TBL instructional strategy, and were also using IF-AT technology that CTLA supplied for faculty using this strategy. Five faculty from those groups have implemented TBL and use IF-AT technology to support students’ learning, active student engagement, and immediate feedback to students for Readiness Assurance Test answers.
To assess that faculty use resources provided by CTLA to improve their teaching, CTLA used two measures. It assessed its Faculty Learning Community program with a survey for faculty participants using a modification of the assessment questions from Milton D. Cox, Miami University, Ohio, © 2008. Both program components and developmental outcomes were assessed. Faculty responded that the two program components of highest impact were the colleagueship and learning from the other community participants and the facilitation of the group. The two highest impact developmental outcomes were their ability to select significant learning outcomes for their courses and their interest in the teaching process. Also to also assess resources provided by CTLA being used by faculty, CTLA drew upon usage records to assess that 27 resident faculty used the CTLA lab for a range of 1 to 8 visits for the year, and 15 adjunct faculty used the CTLA for a range of 1 to 26 visits a year.
Link to Full Assessment Reports
2007-2008
Link to Full Assessment Reports
2006-2007
Link to Full Assessment Reports
2005-2006
Link to Full Assessment Reports