Scaling Reading Comprehension Content With Editor-Guided AI
Protagona partnered with a leading non-profit EdTech provider to build a production-grade AI content generation platform for K-12 reading comprehension, embedding a structured editorial feedback loop that lets curriculum specialists drive quality improvements without developer intervention.
Industry
Nonprofit
Teams & Services
Back-End Engineering, AI/ML, DevOps, Solution Architecture, Technical Project Management
Tech & Tools
AWS Lambda, Amazon S3, Amazon DynamoDB, AWS CDK, Amazon SQS, Claude (Anthropic), S3 Vector Storage, JSON-based In-Memory Retrieval, REST API
Key Data Points
The Vision
A leading non-profit provider of free K-12 reading comprehension content recognized that manually producing and refining educational materials at scale was unsustainable as demand for personalized, standards-aligned content grew. Having established foundational AWS infrastructure through an earlier proof-of-concept, the organization was ready to move from experimentation to production — transforming how curriculum specialists interact with AI-generated passages, questions, and answers at scale.
The Goal
Protagona aimed to deliver a production-ready AI content generation system capable of producing grade-appropriate reading comprehension questions and answers across multiple content types. A structured feedback mechanism would allow curriculum editors to guide and refine AI outputs iteratively — without requiring developer intervention at every step — giving content teams direct control over generation quality.
The Challenge
The core technical challenge was building an AI pipeline that produced educationally sound, curriculum-aligned content reliably across repeated iterations with evolving editor feedback. LLM outputs are inherently non-deterministic: early iterations showed that applying new feedback could inadvertently revert previously accepted improvements, creating regression cycles that frustrated reviewers and slowed quality convergence. Designing a prompt architecture that preserved accepted changes while incorporating new guidance — without inflating context windows to the point of degrading model performance — required careful experimentation and architectural judgment.
The Solution
Protagona designed a content generation system that treats each reading comprehension content type — grade-adjusted passages, vocabulary questions, author's craft questions, and others — as its own independent unit, each with its own tunable guidance. This let curriculum specialists shape how the AI generated content for each domain directly, without needing a developer to make code changes.
A feedback API closed the loop between editorial judgment and AI generation, giving content editors a direct channel to submit guidance into the system. Earlier iterations had run into a regression problem, where new feedback could cause the AI to lose previously accepted improvements. The team solved this by designing the system to retain a history of prior editorial decisions, weighing new feedback against what had already been approved so quality consistently moved forward rather than circling back.
.png)