Why ChatGPT can’t create a residency schedule

Picture this: It's 11 PM on a Sunday night. A chief resident is hunched over their laptop, Excel open, multiple browser tabs pulled up, and WhatsApp messages from residents piling in. They've already spent six hours trying to balance night float, vacation requests, and clinic coverage. Out of desperation, they type into ChatGPT:

"Make me a compliant call schedule for 17 residents over 4 weeks. Constraints: no more than 80 hours per week, one day off in seven, equal weekend distribution, respect vacation requests, balance nights fairly, etc." (There are a lot more rules than these!)

ChatGPT whirs. A minute later, a polished-looking table appears. For a brief, shining moment, the chief resident thinks: I've been saved.

Then they look closer.

• One intern is scheduled for five consecutive 24-hour shifts.

• Another resident is double-booked for clinic and night float on the same day.

• Two vacations are ignored entirely.

What looked promising is, in reality, an unusable schedule.

So she tries again. Different prompts. More details. She even upgrades and buys ChatGPT Plus, hoping the extra horsepower will finally crack the problem. But she quickly discovers she's not alone — other chiefs are doing the same thing, shelling out for Plus in search of a miracle fix.

And yet, no matter how much they tweak, re-prompt, or pay, the outcome is always the same: a schedule that looks neat in theory but falls apart the moment you check the details.

The Reality of Residency Scheduling

Every residency program has someone carrying the burden of creating schedules — usually a chief resident or an admin staff. It's a responsibility that can consume 10+ hours per week, on top of clinical duties that already exceed 80 hours.

Residency schedules aren't just about filling boxes. They're about ensuring a safe level of care, respecting your colleague’s wedding, and seeing that patient in the continuity clinic. A valid schedule must satisfy:

• Hard rules: duty-hour limits, continuity clinic requirements, vacation blocks

• Soft rules: fairness, equity, balanced learning opportunities

• Dynamic changes: swaps, sick leave, sudden staffing gaps

When chiefs saw the hype around AI, many thought: finally, a way out.

Another chief laughed when I asked if they'd tried it:

"Yeah, I bought the Plus, wrote out all the rules and had a long chat with it, and then it finally said: ‘This is beyond me.'"

Why ChatGPT Struggles

Before we move on — ChatGPT is the app you use, while a large language model (LLM) is the technology powering it behind the scenes. Think of an LLM like a really sophisticated prediction engine that has read millions of books and learned which words typically follow other words in different situations — it's built on a neural network (kind of like an artificial brain) that creates billions of connections between words and ideas, so it can predict what should come next in a sentence based on patterns it's seen before. That's why it's great at writing things that sound right and follow common patterns (like emails or patient summaries), but residency scheduling requires following a bunch of strict rules at the same time. This is a constraint optimization problem, closer to math than prose.

This isn't a knock against AI. ChatGPT can explain the rules of scheduling, suggest frameworks, or even write code. But when asked to produce a final, working schedule that balances hundreds of moving parts, it fails.

So, What's the Solution?

The problem isn't that AI can't help. It can help — just with a little more support than the normal app. Residency scheduling is about solving a complex puzzle with lots of rules, needing “optimization” tools - which are mathematical algorithms to figure out the best way to arrange everything whilst following the rules.*

The sweet spot may be first fine-tuning a generic LLM. Then, let the fine-tuned LLM understand your scheduling constraints. Next, the fine-tuned LLM becomes an expert programmer that sets up the mathematical algorithm and runs it to produce a schedule. In other words, it essentially ingests your residents, vacation, rules, and converts them into code for the optimizer to generate a schedule. Finally, it can draft emails for announcements and coordinate swaps within the schedule.

What’s next?

Residency scheduling is one of the most thankless jobs in medicine. Chiefs are right to hope for relief, but ChatGPT isn't the savior — at least not yet.

As LLMs are getting better at coding every day, we can have hope that more tools will be available for Chiefs to solve their scheduling pains.

*If this interests you — there’s actually a whole field of research called “Operations Research” that studies how to use mathematical algorithms to divide scarce resources (like physicians’ time) in the best way possible. In fact, there’s a specific problem in this field called the “Nurse Scheduling Problem” that focuses entirely on optimizing healthcare workers’ schedules.

Small plug: At Standard Form, we’re building tools to take the grunt work out of residency scheduling. With a little sprinkle of AI, we cut scheduling time by 50% or more — without the chaos. Interested? Join our waitlist.

Thanks to Owen Kosman and Shreya Dristi for reading drafts of this.