The overall evaluation of this project will include peer and student assessment of the written materials, assessment of student learning, student attitudes and student process skills. The formal assessment will be conducted by the consortium members and formally identified beta testers. We welcome any additional assessment data collected by instructors who are using the materials. is leading the assessment of the project, and should be contacted for additional information regarding assessment.
FORMAL: Data collected by Alpha and Beta testers for analysis and dissemination to include:
-
Student rubrics on Specific Activities:
Students in selected classes will complete a rubric assessing the effectiveness of the activity in meeting the content and process outcomes. They will also be asked to comment on clarity and suggest improvements. -
SALG (Student Assessment of Learning Gains) survey:
The Student Assessment of their Learning Gains (SALG) is designed to be a more useful course assessment instrument. The SALG allows instructors to gather learning-focused feedback from students. The SALG survey asks students to rate how each component of a course (e.g., textbook, collaborative work, labs) helped them to learn and to rate their gains toward achieving the course goals. The SALG is web-based and can be found at http://www.salgsite.org/. -
In-Class Exams:
These are the normal exams faculty administer throughout the course. There will be some common questions that are generated by the faculty participating in the project to allow comparison among semesters and institutions. -
ChemX survey:
Nathaniel Grove and Stacey Lowery Bretz, "CHEMX: An Instrument To Assess Students' Cognitive Expectations for Learning Chemistry," Journal of Chemical Education, 84(9), 1524-1529, 2007. CHEMX consists of 47 statements divided into seven clusters (effort, concepts, math link, reality link, outcome, laboratory, and visualization) and uses a standard five-point Likert scale. -
ACS Quantitative Analysis Exam:
The exams are nationally normed and provide a basis to compare performance of students in the POGIL classrooms to those nationwide. -
Student Interviews and site visits:
Students will be given an analytical chemistry problem to solve. This problem will be determined by the course instructor and the investigator based on what concepts have been covered in class at the time of the interview. Students will be asked to "think aloud" as they complete the problem. The intent of the interview is to elicit information on how student solve problems - What approaches and patterns do they use? How do they think about problem components? Do they validate answers? etc. -
Faculty Journaling and Activity Reviews:
Participating faculty will be asked to keep a journal of classroom observations and note the impact of POGIL activities on their students and classroom dynamics. They will also complete assessment surveys regarding the effectiveness of workshops and the consortium model. As activities are developed, consortium members peer review the activities to ensure consistency and quality before they are classroom tested.
INFORMAL: We would be happy to have all interested instructors contribute to our ANA-POGIL assessment efforts in any of the following ways:
-
Activity Reviews:
We want your feedback on the activities themselves and your classroom implementation. You can complete a review form online for any activity you use. You have the option of simply sending us your information to be used in revising the materials, or sending us your info and allowing us to include the information in our research and dissemination efforts. Your identity will be strictly confidential in all formal dissemination. Reviews posted online can include your name or be posted anonymously. -
SALG:
You may want to use the SALG as part of your course evaluation to gain more insight into student perspectives of their learning. The SALG can provide information on how students perceive the impact of different learning activities and how much progress they have made toward course goals. The SALG is generally given towards the end of a course. -
ChemX:
CHEMX can detect gaps between faculty expectations for learning and those of students, as well as measure changes in students' cognitive expectations across the chemistry curriculum. -
ACS Exams:
The exams available to chemistry teachers include more than fifty exams covering general chemistry, organic chemistry, analytical chemistry, physical chemistry, inorganic chemistry, biochemistry, polymer chemistry, and high school chemistry. (http://www4.uwm.edu/chemexams/index.html) -
Any other methodologies you might employ:
Do you have any assessment tools you use in your class that would provide us with insights into our materials and their use? If so, and you're willing to share, please let us know.
A NOTE ABOUT CONTRIBUTING YOUR ASSESSMENT DATA TO THE ANA-POGIL PROJECT:
Anyone may choose to implement any of the listed assessment tools in their courses. We are interested in hearing about the outcomes of any of your assessment of the ANA-POGIL materials, and we can use your feedback to improve the quality of our materials. If you would like to formally contribute to the research and dissemination efforts of the ANA-POGIL project or present your results in a public forum, you will need to obtain IRB approval before data collection.
Contact with questions or for additional information.