AAI_2025_Capstone_Chronicles_Combined

19

Table 2 Comparison of clustering model results

Cluster 1

Cluster 2

Cluster 3

Cluster 4

Accuracy Run Time

DEC

Step-By-Step Guided Prac tice

Addressing Errors and Building Un derstanding

Extended Practice Across Topics

Diverse Problem Solving Approaches

55% 2m13s

K-Means

Practice Prob lems

Geometry

Trigonometry Calculus

76% 3s

LLM-as-Classifier Arithmetic

Geometry

Algebra

Calculus

87% 15m32s

The “Process-Oriented Engaged Learners” cluster groups together 62 users who are focused on understanding mathematical processes and concepts. “Foundational Skills Learners” consisted of 45 users who were working on basic mathematical concepts with positive attitudes. The remaining conversations were captured in “Mixed Engagement with Learning Challenges.” These users displayed varied emotional states while actively participating in learning. Conclusions Ultimately, deep embedded clustering presented significant challenges for our use case. The DEC model was much more difficult to design and tune than the k-means and LLM-as-classifier models, required far more computational resources, and yielded less useful results. However, it did often create interesting and nuanced clusters, and was likely inhibited by the small dataset relative to the typically large data requirements of deep learning models. On large datasets, DEC could be a valuable option when deep contextual clustering is needed. Although the DEC model was not reliable enough for use in the AI tutoring application, the use of clustering in this context still showed great promise, particularly when combined with large language models. The LLM-as-classifier model was able to significantly outperform both the k-means and DEC models, and the generated clusters were able to be incorporated into the tutoring application in a way that created an intuitive data explorer for users. The use of unsupervised learning presents major advantages for application maintenance over supervised techniques, as it allows the system to automatically generate labels based on the course content without any human intervention or manual labeling. The incorporation of facet extraction also

243