Case Study: Rethinking Student Writing Skills in the Age of AI
Share
Category
AI Tools
Preview link
PDFAthena AI at St. Catherine's Academy
St. Catherine's Academy, a Catholic all-boys school in Anaheim, California, successfully transformed student writing instruction by implementing Athena AI to address the growing gap between AI usage and critical thinking skills. In early 2025, the school discovered that 62% of students were using AI writing tools, yet 89% couldn't critically edit or improve AI output—a pattern reflecting national trends.
The Challenge
The Hidden Risk of AI in Education:
- 62% of students leveraging AI writing tools for assignments
- 89% unable to critically edit or improve AI output
- Writing assignments completed faster, but genuine learning and critical thinking declining
- Teacher fatigue preventing adequate intervention at scale
- Academic performance plateauing despite technological assistance
School Context
St. Catherine's Academy Profile:
- Founded: 1889 (136 years of educational excellence)
- Type: Catholic school with military tradition
- Student Body: 163 all-boys students (K-8)
- Demographics: 88% students of color
- Faculty: 17 teachers with 64% holding advanced degrees
- Student-Teacher Ratio: 10:1 (significantly better than national average)
- Academic Standing: Top 20% of private schools in California
The Athena Solution
"If AI is the calculator, Athena is the math teacher"
Core Components:
- Embedded Scaffolds: Targeted questions throughout writing process ensuring drafts are revised and improved
- Forced Reflection: Assignments requiring students to justify edits and explain changes
- Process Visibility: Full tracking of work evolution from AI output to polished submission
- Metacognitive Tracking: "Thinking about thinking" to help students articulate strategies and recognize strengths
Implementation Results
Student Outcomes:
- Student engagement: 32% → 57% (+78% increase)
- Average math and writing scores: +14% improvement
- Assignment completion rates: 68% → 83% (+22% increase)
Teacher Benefits:
- Weekly administrative hours: 67% reduction
- Teacher retention: +9% improvement
- Qualitative shift: Students reported feeling "seen" in their learning journey
Educational Impact:
- Students developed critical AI literacy skills
- Enhanced metacognitive awareness and self-regulation
- Improved writing process visibility for both students and teachers
- Transformation from passive AI consumption to active, thoughtful engagement
The Broader Context
National Trends Supporting This Initiative:
- 66% of students globally use ChatGPT for coursework
- 89% of students admit to using ChatGPT for homework
- Teen usage for schoolwork doubled from 15% (2023) to 30% (2025)
- Only 46% of students report feeling engaged at school (down from 65% in 2018)
- 70% of educators overwhelmed with grading and administrative tasks
- 40% of teachers report being "always" or "very often" burned out
Key Success Factors
What Made the Difference:
- Teacher-Centered Design: Positioning AI as support tool, not replacement
- Comprehensive Training: Professional development emphasizing ethical AI use
- Gradual Implementation: Phased rollout allowing for adjustment and feedback
- Focus on Critical Thinking: Emphasizing evaluation and improvement of AI output
- Metacognitive Development: Building students' awareness of their own learning processes
Future Plans
St. Catherine's Academy is expanding the program with:
- Enhanced multilingual support for EAL students
- Parent dashboards with AI-generated progress summaries
- AI-driven individualized education plans (IEPs) for students with disabilities
- District-wide scaling for 2025-26 school year
Lessons Learned
Critical Insights:
- Simply providing AI access isn't enough—students need explicit instruction in critical evaluation
- The real transformation occurs when students learn to question, critique, and improve machine output
- Technology implementation requires comprehensive teacher support and training
- Metacognitive skill development is essential for meaningful AI integration
- Small school environments can be ideal testing grounds for innovative educational technology
Conclusion
St. Catherine's Academy demonstrates that with thoughtful implementation, AI can become a powerful catalyst for deeper learning rather than a shortcut that bypasses critical thinking. The school has successfully transformed a technological challenge into an educational opportunity, creating a new generation of thinkers rather than passive technology consumers.
This case study reflects authentic implementation data from St. Catherine's Academy's Athena AI pilot program during the 2024-2025 school year. All outcomes represent verified measurements from the school's assessment and evaluation systems.


Athena lets teachers use AI to personalize learning.
Design AI to tutor or assess students in 1-1 conversations, based on custom learning objectives and materials.
Get a Demo
