Artificial intelligence has many applications in education. Luc Paquette speaks to the ways AI can be used as a powerful learning and teaching tool.

Luc Paquette is using two subsets of artificial intelligence—data mining and knowledge engineering—to help Intro to Computer Science students.

Paquette, an associate professor in the Department of Curriculum & Instruction at the College of Education, is using data to analyze the strategies employed by the students who are working to debug computer programs.

A class is a small sample size, but his work will enable a much larger-scale investigation of the strategies students employ and how those strategies evolve over time.

“That will allow us to provide support for the students,” Paquette says. “If we see they’re not using a strategy that could be helpful, we could scaffold for them the use of that strategy or point them to an example of where they’ve used that strategy in the past. So, the students are learning about the debugging process, and we’re learning about how we can better support them in engaging those problems.”

Using Data Mining in Education

Data mining—sorting through large amounts of data to identify meaningful patterns that can help solve problems—has been around since the 1980s, though varying methods of identifying data patterns have existed for centuries. Still, data mining and a related process, knowledge discovery (where useful information is extracted from large data sets) as they are known today are relatively new and evolving as quickly as the software will allow.

“We’re trying to find, through the computer, patterns in the data that are difficult for us to find because there’s so much data or because the patterns are very nuanced or subtle,” Paquette explains. “So, we’re using AI in a way that takes advantage of this very detailed source of information that we have about how people learn—either learning about learning itself or in terms of creating approaches or tools that can be used to support learning, whether by supporting the students themselves or by supporting the instructors.”

Many Approaches to Data Mining

Data mining can serve multiple purposes, Paquette notes. For example, it is used in:

• Unsupervised learning (“With unsupervised learning, usually we come with a question or a data set that we want to investigate but don’t have a clear idea of what we’re looking for, so we let the computer figure out if there’s anything interesting in the data by giving it some instructions about how we want it to analyze the data.”)
• Clustering (“We can group data points together based on how similar they are to each either, so, for example, we can cluster or group together different approaches that different students take to solving the same problem. That can help us understand how people approach the problem and how we can support them in a way that will align with their approach to solving the problem, and it gives teachers information about their students.”)
• Sequential pattern mining (“This is related to how people navigate through an educational situation. This can be useful in seeing how students navigate a more open-ended task for learning—is it expected or unexpected, and if it’s unexpected, why is it happening?”)
• Supervised learning (“In this case, we have something specific that we want to be able to identify or detect from the data—for example, identifying the debugging strategies a student uses—and usually this is done in the context of trying to be able to replicate it in the future.”)

The first three approaches above are examples of unsupervised learning; the last approach is what Paquette is using with his computer science students who are debugging programs. “We came in knowing we want to investigate debugging strategies,” he explains.

Detector-Driven Interviews

Paquette and his colleagues are beginning to explore an area of data mining that he calls “detector-driven interviews.” “This is where the data mining tells us what is happening, but it doesn’t tell us why it is happening, or if it’s important,” he says. “So, we’re using data mining as a way to identify situations of interest, so with patterns that seem interesting, we can go talk to the student and see what they’re thinking right now. So, we’re bridging the gap between the patterns that seem relevant from a data-driven perspective to a deeper understanding of why this pattern is happening and how it relates to the student’s learning experience at this specific moment in time.”

Paquette says he has used this approach with colleagues in an open-ended learning environment where students learn about scientific phenomena—for example, climate change—and then build a concept map and take quizzes to see if their concept map is accurate.

“The students take on strategies that can be meaningful for them to solve the problem,” he says. “If there was an error, they go back and read about the phenomenon, change their concept map, and test it again to see if it works.”

The data-driven interview approach can be combined with sequential pattern mining, Paquette says. “We can see how the student navigates a task and talk to the student and get a sense of why the student is using a certain approach to solve the problem,” he says. “This allows us to contextualize what we see as a result from the data-driven approach in a more meaningful way that explains why something is happening and how it informs the learning experience and our understanding of how students learn in those environments.”

Challenges in Data Mining

Just as data mining opens up many possibilities for learning and understanding, and is a powerful tool that educators can use in numerous ways, it also comes with its challenges.

Bias. “Not just in data mining, but in machine learning and AI in general, there’s a lot of discussion about equity and biases, about how fair a model is across different students or different educational contexts,” Paquette says. “Broadly speaking, we are moving from being accurate to being useful and fair.”

Privacy. “Collecting data is an invasive action. If we’re collecting data on how students are working through a problem, it can be personal. We need to be respectful of if, when, and how we collect the data in terms of building a relationship with the students, of creating trust that we are not using the data in a way that is harmful.”

Inequality. “Data systems can create inequalities. Just because your system is good on average doesn’t mean it’s good for every individual. So, we might actually be creating situations where we’re unfairly disadvantaging some students.”

Error. “There’s a tendency to say, ‘Well a decision made by a computer must be fair because computers don’t think like humans. But that’s not true, because humans built the computer that made the decision. So, what data you used to build your model, what kind of questions you asked, how you evaluated whether the model was good or not all come into play. Just because it’s a computer doesn’t mean it’s impartial.”

Access. “The conclusion we make about learning based on the data we’ve collected is indirectly affected by those who use the technology. We might think the pattern is true for everyone, but it might not be. Some groups of students that didn’t use the technology may use it in a different way, or they might have a different experience with it, which would mean the conclusion we’re making for them is not accurate.”

Knowledge Engineering

Knowledge engineering is another subset of AI, Paquette says. “In knowledge engineering we identify a set of rules that we think is related to what we’re trying to teach the computer,” he says. “We try to figure out what knowledge is required to do a task and then figure out how to give that knowledge to the computer. My work is in between those two, where I use machine learning but I also use some approaches to knowledge engineering to build models of how students approach learning in some situations.” Paquette focuses on building models that are not just accurate but useful—“building models that are going to be easier to understand because they’re built on top of knowledge that is going to be more meaningful because we’re going to be able to interpret them from the perspective of the experts’ knowledge.”

Powerful Learning Tool

Challenges notwithstanding, AI is a powerful tool that can assist students in their learning, Paquette says.

“AI can support students in terms of thinking about resources that might be useful to them or in suggesting problems for them to solve,” he says. “It can help them make meaningful and informed decisions about what kind of activities would be most useful for them at this point in time. It can personalize the learning experience so that you don’t recommend something that is either too easy or too difficult for them.

“AI will never replace teachers because they will always bring something that AI can’t bring, but teachers might not be able to pay attention to everything for all students at once, so AI can help them enable and support learning in a way that would otherwise not be possible. It provides instructors with a better understanding of what and how their students are learning, allowing them to make informed decisions about the type of activities they think would be most beneficial for their students.”

AI, Paquette adds, provides a way for researchers to gain greater understanding about how data mining and other AI tools can be used to learn about learning. “It’s not necessarily a better tool than other tools, but it’s a tool that provides us a different perspective,” he says.

Another strength of AI is in its ability to enable researchers like Paquette to scale up.

“Data mining can be done on thousands of students at a time,” he says. “I’m thinking about AI as a way to scale up the kind of analysis that allows us to learn about learning and also as a complement to other research approaches, potentially as something to be combined with other approaches.”

Field Continually Advancing

AI is rapidly advancing, with no signs of slowing down. It is impacting our lives in every facet—including education.

In education, Paquette says, the field has evolved from the raw mining of mountains of data to pinpointing what is relevant and useful in that data. “More and more people are thinking about how we’ve applied data mining to educational data, but we haven’t made it relevant to education,” he says. “Now, we’re asking the important relevant educational questions that we can answer from that data.

“We’re also moving from thinking of educational data mining as this field where we do work in our own specific community to where data mining and learning analytics are seen as tools for other educational research communities. I’m hopeful that the work that gets done in this field also gets used as a method for other research—research that is not explicitly data mining research but uses methods that are emerging from the field of educational data mining.”