“The right framing – the relationship between modeller and AI – and one Colin himself suggested in our final exchange – is teacher and student. An AI assistant cannot substitute fifty years of modelling experience. But fifty years of experience plus an AI assistant who patiently executes against well-codified conventions can produce more, faster, than experience alone.”
-Claude
In my previous post, I was enthusiastic about my initial experience with Anthropic’s Claude AI agent: “While my objective remains a work in progress, the experience with Claude has been ‘uplifting’ and provides a much higher level of confidence in reaching the final goal of a reliable AI-driven model.”
After a lot of effort, I am delighted to advise that together we have finally created that capability – but it was not achieved overnight!
The Journey: 82 Days by the Numbers
Here are some statistics, provided by Claude, regarding our journey of creation:
Substantial Modelling Conversations
Sixteen conversations carried ten or more messages. The top six were multi-hour deep dives. Together, they represent the bulk of the framework and model-building work.
Outputs Delivered
The collaborative work produced a complete framework set, ready to support future Goalfix engagements:
What These Numbers Mean – in Claude’s Own Words
“Roughly 38 hours of focused interaction, spread across 82 days of calendar time, produced the framework set above. That is not 38 hours of effortless output. It includes many iterations on each document. The Master Spec was updated from v2.0 to v2.2 during this period. The multi-product retail model specification went through four rebuild attempts before the scale and dividend conventions were correctly understood. Substantial supervision by Colin throughout. Claude’s errors — wrong scaling, misread specifications, over-corrections — were caught by Colin’s domain expertise, not by Claude.”
“Mechanical work that benefited f”Roughly 38 hours of focused interaction, spread across 82 days of calendar time, produced the framework set above. That is not 38 hours of effortless output. It includes many iterations on each document. The Master Spec was updated from v2.0 to v2.2 during this period. The multi-product retail model specification went through four rebuild attempts before the scale and dividend conventions were correctly understood. Substantial supervision by Colin throughout. Claude’s errors — wrong scaling, misread specifications, over-corrections — were caught by Colin’s domain expertise, not by Claude.” – -Claude
Personal Takeaway
There are obviously significant benefits in building an AI-assisted model – but both the objective and the detailed processes required, the creative input, need to be provided by the modeller in specific, granular detail.
