The Power of Many Small Practice Problems in Engineering
Dr. Matthew Liberatore is a professor of chemical engineering at the University of Toledo and the author of the Material and Energy Balances zyBook and other titles. Professor Liberatore has conducted extensive research into how students absorb engineering learning and shares his insights here on the power of many small practice problems.
It seems obvious that the more practice problems you can give students, the better. Or is it?
Professor Liberatore Yes, but with some caveats. More practice problems is one of the most common things a student will ask a professor, especially an engineering professor. Traditionally, you have a big, fat engineering textbook with 20, 50 or even 100 questions at the end of a chapter.
But do all these problems give students satisfaction?
Students see a bunch of big paragraphs with a question about a particular chemical reaction. The next question is kind of the same, but about a different chemical reaction. The quantity is there, but a combination of procedural or “plug and chug” problems, and problems with different levels of conceptual understanding, are needed for learning and becoming a better engineering problem solver.
Professor Liberatore’s homework success menu
- Lots of spaced-out, regular practice
- Instant feedback
- Auto-grading
- Formative and summative problems
How do you improve these practice problems?
Dr. Matthew Liberatore There are a number of approaches.
First, there’s feedback. Students typically receive feedback during office hours (or as we call them, student hours). But in bigger classes – 100 or 200 students, or more – I don’t have time to give each student feedback unless the problems are auto-graded.
Second, you can randomize each question individually. Rather than just change a chemical reaction, you can change multiple inputs, like temperatures and pressures, as well as combinations of chemicals. Now, the problem is different enough.
And third, offer a mix of both formative and summative practice problems.
What do you mean by formative and summative problems?
As an engineering professor, I think of formative questions as exercises that literally help your students form knowledge — specifically they remember and understand levels of Bloom’s Taxonomy. Students focus on one concept at a time, the problems are scaffolded from easier to more challenging, and there’s no penalty for wrong answers.
If I can use a basketball analogy, these formative exercises are teaching you the individual skills, like dribbling and passing, doing your defensive slide, and getting into your stance. These formative exercises are teaching you the individual abilities you need to become a better basketball player.
But you also need to actually play the game. That’s the summative problem.
Playing the game as an engineering student means tackling those hard problems they find on exams and moving toward real-world engineering problems. These summative problems tie together multiple concepts, are about a paragraph long, and have some kind of narrative. We usually see these problems at the end of chapters.
In my Material and Energy Balances zyBook, we include over 400 formative questions and hundreds of summative assessments. Most questions are structured as “challenge activities,” as we call them. All these questions are autograded, and students can work on the challenge activities over and over until they get them right, essentially creating perpetual practice.
Formative and summative problems are both important. I’ve led research demonstrating that working on them closely correlates with student success, but in a surprising way.
(Courtesy of Dr. Matthew Liberatore)
(Courtesy of Dr. Matthew Liberatore)
How do formative and summative problems correlate with student success?
Dr. Matthew Liberatore As the author of engineering textbooks, I became very curious about this question, and working with some undergraduate researchers, we tried to see if any correlations existed.
To our surprise, we found that the easier problems, those formative ones, the dribbling and passing drills, more strongly correlated with the final course grade than the summative, end-of-chapter problems. We ran studies over a couple of years, and the results were almost identical – even though the summative assessments were two completely different sets of problems.
Why are formative problems so important?
Dr. Matthew Liberatore We found that students practicing with small problems really gives them skills and confidence that should translate to the bigger, more complex questions.
This is the hallmark of engineering, right? Being able to break down big, complicated things like, say, a chemical plant, into smaller pieces.
Those smaller problems give students the opportunity to exude some curiosity, too. In some problems, you can change the situation with each attempt. For example, you may be asked about data related to different chemicals, like benzene or water or acetone, in order to make other calculations that are important to the mass balance, the energy balance, or other things that we’re working on.
The little pieces add up, helping students build the foundational skills that help them build the bigger skills they need to become successful engineers.
What about summative problems?
Summative problems combine the many concepts and skills within one or several chapters.
Returning to the basketball analogy, the individual skills of each player need to be pulled together to execute a play correctly and score the winning basket: One person drills and passes, another sets a screen, and yet another takes the last shot.
In engineering, those individual skills, concepts, and calculations translate into solving a problem of taking raw materials to value-added products. Our students have reverse-engineered problems from YouTube videos, including “Show How It’s Made,” over many years. For example, how you make soda or pop (word choice depends on where you are from) starts with the raw materials like water, sugar, and carbon dioxide and ends with the canned or bottled beverage.
Summative problems do correlate with students’ final grades as well, just not as strongly as the formative problems.
(Courtesy of Dr. Matthew Liberatore)
How much formative practice do students need?
Dr. Matthew Liberatore Looking at large datasets of hundreds of formative questions being solved by hundreds of students over many years, we found that the median B student completes 95% or 96% of the problems, which is great. However, at least half of the A students complete 100% of those formative problems. So the A students are doing more of that dribbling and passing, working on the fundamentals. The difference is subtle but that deliberate practice counts.
The bigger variances appear between B and C, and C and lower students. They’re just getting a lot less correct. At this point I can tell if a student is in trouble if they’re completing less than 80% of the problems, since 75% of C students correctly solve over 80% of the challenge activities.
Final thoughts?
Dr. Matthew Liberatore Critical thinking and problem solving skills are central to engineering. The problems you face as an engineer are hard. The more practice that I can give my students to develop those skills, the better prepared they are to translate the skills to real world problems.
Related research papers
Can I Have More Problems to Practice? Part 1
Can I Have More Problems to Practice? Part 2
Reading analytics and student performance when using an interactive textbook
