Why I Chose Automatic Question Generation: The Story Before the Thesis

 

    This is the first in a series about my master's research on AI question generation for scientific reading comprehension. Before I get into what I built or what I found, I want to tell you why. And the reasons go back a lot further than the thesis itself.

It started with a teacher, not a technology

I still remember a science teacher I had as a child who always started class by asking us about the last lesson.

Not a test. Not a quiz for marks. Just questions. "What did we say happens when…?" He would throw them out casually, almost like he was thinking aloud, and then wait for us to dig the answer back out of our own heads.

And somehow, because he made us retrieve it rather than just re-read it, that knowledge stuck. Years later, decades later, I still have it.

I didn't have a word for what he was doing back then. Now I do. It's called active recall. And in a quiet way, that classroom is where this whole research journey begins.

The dream is older than the AI

Here is something people are often surprised by. The idea of building tools for education has been with me for about ten years, ever since I was an undergraduate. It wasn't originally about AI at all. It was simpler than that. I wanted to build things that help educational institutions, schools, tuition centres, and tutors do their work better.

The what stayed remarkably consistent for a decade. Only the how kept changing, as the technology slowly caught up.

The teacher's real problem: I was drowning in questions

For eleven years I tutored students. First in Sri Lanka, and later online across the UK, Germany, the UAE, and Southeast Asia, preparing them for Cambridge exams in maths, chemistry, physics, and computer science.

And if you have ever taught, you will know the quiet, relentless workload hiding behind it. Writing questions. Endless questions. Practice sets, mock papers, quick checks for understanding. It ate an enormous amount of my energy.

So I kept thinking to myself, what if a tool could generate questions the way I do? Not to replace my judgment, but to lift the mechanical load off my shoulders, so I could spend my energy on the part that actually needs a human being. The teaching. The innovating. The building of better ways to learn.

That is a very unglamorous, very honest origin for a research topic. It came from being tired.

Version zero: predicting the exam paper

My first real idea for this came back in 2020, before ChatGPT existed, and it wasn't a chatbot at all. It was a prediction tool.

The plan was to feed in years of past Cambridge papers, let the computer find the patterns hiding inside them, and then forecast the questions most likely to come up in future exams. I had studied the fundamentals of AI and deep learning, including an artificial intelligence course on Coursera taught by a professor who helped pioneer the field at Stanford, so it felt within reach.

And it wasn't only theory. I had already built something structurally similar with a friend. A MATLAB project that used around twenty years of historical UK land temperature data to predict future temperatures. Same instinct. Patterns in, predictions out. Applying that same logic to exam questions felt like a natural next step.

Then the world changed.

Germany, ChatGPT, and everything happening at once

I moved to Germany in 2022 to go deeper into educational technology, to really understand what kinds of tools could genuinely serve learning. And almost on cue, ChatGPT went viral.

Suddenly the thing that would have taken me months of deep learning work was far more achievable. I threw myself into learning how to build applications with large language models and their APIs, and specifically into something called retrieval augmented generation, or RAG, because grounding a model in real study material was exactly what education needed.

Everything was happening in parallel. A decade old dream, a teacher's workload problem, a new country, and a technology that had just made the impossible feel easy.

The moment it became concrete: Owl Mentor

The idea finally crystallised in a course called Multimedia Learning. My lecturer and his assistant were working on an app, which I will call Owl Mentor, where you could simply chat about your study materials. You could have a natural conversation with an AI about a chapter, and pull the knowledge out through dialogue.

It hit me straight away, because I had been quietly carrying almost exactly this idea for my own students back at my tutoring company in Sri Lanka. Seeing it real, working, right there in front of me, that was the thing that turned "someday" into "this is my thesis."

My research became a design based project. I worked closely with my supervisor, brainstorming and shaping what the app should be, while he did the main engineering, and together we improved it in cycles.

What made it mine, and not just a wrapper around ChatGPT

This is the part I care about most, because it is genuinely easy to slap an LLM behind a text box and call it an education tool. What made this different came from two layers of expertise that the model simply does not have on its own.

The first was careful direction. We tested many prompts to control exactly what kinds of questions the model should generate, and, just as importantly, to stop it from hallucinating. Left unguided, an LLM will confidently produce fluent nonsense. Reining that in was real work.

The second layer is my genuine edge, and it is assessment theory. My first master's was in education, where I studied the theory of assessment. How to build a proper test. How to design a question so a student can actually demonstrate their level of learning. When to reach for multiple choice, when for short answer, when for a full essay.

So we did not just ask the model for "questions." We fed it rubrics. The same formal assessment rubrics I would use as a teacher. Those rubrics steered the model toward what a good answer should look like, not merely what a question should look like. A pure developer would rarely think to do this. It is the exact point where a decade in the classroom mattered far more than the code.

Closing the loop: a mirror for the student

In the end, the tool generated three types of questions. Multiple choice, short answer, and the longer reflective kind where a student can really show what they understand.

But my favourite part was what happened after a student answered. The tool would evaluate their response against a generated model answer, show them what earns more marks, and reveal the rubric itself.

That means a student sitting alone, late at night, can genuinely reflect on how well they are doing. They can feel the difference between "I read it and it felt familiar" and "I actually understand this." The rubric stops being just a tool for steering the AI, and becomes a mirror handed to the learner.

Why it all connects

Looking back, the through line is almost embarrassingly consistent. A childhood teacher showed me that retrieving knowledge is what makes it last. A decade as a tutor showed me how much energy question writing quietly devours. A course showed me it could actually be built. And a shift in technology finally made it possible.

Automatic question generation was never a topic I picked because it was fashionable. It is the meeting point of everything I had been circling for ten years. Teaching, technology, and the simple conviction that the right question, asked at the right moment, is one of the most powerful things in all of education.

In the next post, I will get into the thesis itself. How it actually works, and what building it taught me that I never saw coming.

Were you ever taught by someone who asked exactly the right question at exactly the right moment, and it stuck with you for years? I would genuinely love to hear about it.