Evaluation Resources
The following resources aim to assist LLL stakeholders in their evaluation efforts by providing links and documents describing evaluation processes and methods. Included are sites that provide a broad overview of evaluation, as well as links that target particular evaluation methods with an emphasis on ease-of-use and practicality. Topics include:
- Evaluation Overview Resources
- Evaluation Methods
- Evaluation project overview
- Student learning outcomes (development/assessment)
- Writing SLOs
- Assessing SLOs
- Data Collection Methods
- Methods overview
- Ethics
- Questionnaire/surveys
- Increasing Online Survey Response Rates
- Interviews
- Focus groups
- Observation
- Document analysis
Evaluation ‘Overview’ Resources
The following sites provide user-friendly guides for many aspects of evaluation in educational contexts. They include overviews of evaluation processes, as well as "quick tips" or short explanations of different evaluation methodologies.
- University of Texas Instructional Assessment Resources (IAR)
- Pennsylvania State University Extension Program Evaluation Site
- University of Wisconsin Extension, Program Development and Evaluation Site
- National Foreign Language Research Center, Foreign Language Program Evaluation Project (NFRLC, FLPEP)
A comprehensive resource for foreign language program evaluation
Evaluation Methods
Below are links to more specific aspects of evaluation, selected from the sites above and elsewhere, with a focus on educational contexts.
Evaluation project overview
- University of Texas (IAR) A step-by-step guide to planning evaluation projects including checklists, worksheets and example evaluation plans.
- Linking the how to the why: Methods decisions in foreign language program evaluation” A brief, useful description of evaluation project planning and implementation. (Yukiko Watanabe and Castle Sinicrope, 2008, NFRLC FLPEP)
Learning Outcomes (Development/Assessment)
- NFLRC/FLPEP collection of SLO assessment websites
- North Carolina State: Internet Resources for Higher Education Outcomes Assessment
- UHM Office of Assessment
- Writing SLOs
University of Texas (IAR)
A guide to conceptualizing and writing student learning outcomes, with additional resources and links.
-
Assessing SLOs
University Of Michigan, College Of Engineering
Description of developing and assessing SLOs from their ‘Handbook on student learning outcomes assessment’
A collection of SLO development/assessment how-to sites.
Has links to various higher education outcomes assessment sites.
Useful suggestions for developing and assessing program-level learning outcomes.
Data Collection Methods
Methods overview
- University of Texas (IAR) A list of data collection methods with descriptions and step-by-step instructions on instrument planning, implementation, data analysis and reporting.
- Penn. State University Extension Program Evaluation Site Links to many different kinds of evaluation data collection methods.
- Overview of common evaluation methods (Sinicrope & Watanabe, NFLRC/FLPEP) A comprehensive list of data collection methods including a brief description of each tool, its purpose, advantages, and challenges.
Ethics
- University of Texas (IAR) A description of ethical issues related to the collection and storing of data from human subjects .
Questionnaire/surveys
- Using Surveys in Language Program Evaluation (NFRLC, FLPEP) A comprehensive treatment of survey/questionnaire methods.
- University of Texas (IAR) A brief description of how to plan, develop, administer, analyze and report data for survey instruments.
- University of Michigan College of Engineering: Assessment for Curricular Improvement How to develop surveys as well as advice on choosing samples (i.e. groups to administer the survey to).
- Creative Research Systems - survey design Description of survey development including tips on ‘dos and don’ts’.
- University of Reading Statistical Services Centre Describes approaches to the analysis of survey data.
Increasing Online Survey Response Rates
- Investigating strategies for increasing student response rates to online-delivered course evaluations (Norris & Conn, 2005) A published in The Quarterly Review of Distance Education.
- University of Texas (IAR) Guidelines for maximizing response rates.
- Online survey response rates and times: background and guidance for industry (Hamilton, 2003) A white paper from a private consultant.
- Advice from the College of LLL Evaluation Resource Team Recommendations for maximizing survey response rates.
Interviews
- Online Advice on Designing and Analyzing Interviews A comprehensive listing of links to interview techniques, hosted on the NFLRC/FLPEP webpage.
- University of Texas (IAR) A brief description of how to plan, develop, conduct and report data for interviews.
- Penn. State University Extension Program Evaluation Site Documents and links on interview techniques.
- Online Evaluation Resource Library: Preparing interviews
- Online Evaluation Resource Library: Administering interviews
Focus Groups
- University of Texas (IAR) Focus group planning, implementation, data analysis and reporting.
- University of Arizona Detailed description of focus group data collection methods.
Observation
- University of Wisconsin Extension, Program Development and Evaluation Site Discusses how to collect direct observation data.
Document Analysis
- University of Texas (IAR) Document analysis planning, data gathering, and reporting results.
Reporting Data
- Gene Shackman's Program Evaluation Resource Page Has many links for presentation/reporting of results.
- University of Texas (IAR) A guide to data analysis and reporting practices for different types of evaluation data.
Using Findings for Program Change
- UHM Office of Assessment Tips for using evaluation results.
- Penn. State University Extension Program Evaluation Site Tips for using evaluation data to write evaluation impact statements and stating program value.
WASC – Using Data for Accreditation Purposes
- UHM Office of Assessment: WASC Guide WASC resource/guide for using evidence in the accreditation process.
- UHM Office of Assessment: Using Rubrics WASC guidelines on rubrics for assessing effectiveness of assessment processes.