Problem framing
Whether this needs a model at all. A ranked query or a set of rules is often better, cheaper and easier to defend.
Automation
Applied AI & decision systems
Scoring, ranking, extraction and retrieval, built so that the output can be explained to the person it affects, and so that a human, not the model, makes the final call.
The situation
The demo was convincing. Then it met reality.
Getting a model to produce plausible output is now easy. Knowing whether it is right, whether it is fair, and what it does on the inputs nobody anticipated is the actual engineering, and it is where most projects quietly stall.
We have built this end to end: a selection system that scores candidates for an education programme. The instructive part was discovering that our first design rewarded a head start rather than measuring distance travelled. Fixing that meant changing the features, not the algorithm.
Scope
What the work includes
Evaluation harness first
A test set and a metric before the model. If we cannot tell you how we will know it works, we should not be building it.
Feature and retrieval design
Structured signals, document retrieval and prompt or model design, chosen for what can be explained afterwards.
Explainable output
Every score arrives with the reasoning that produced it, in language the reviewing human can argue with.
Human-in-the-loop workflow
The interface where a person reviews, overrides and records why, and where those overrides feed back into evaluation.
Bias and edge-case review
Deliberate testing across the groups the system will judge, documented rather than assumed.
5-fold
Cross-validation as a minimum before we report a number
1
Human who holds the final decision, always
0
Automated decisions about people without a review path
Deliverables
What you are left holding
Every engagement ends with artefacts you own outright, whether we continue together or not.
- Evaluation harness and baseline results
- Model or retrieval service with API
- Review interface for human decision-makers
- Written model card and known limitations
Typical stack
What we build it with
Chosen for support horizon and hiring pool, not novelty. If your team already runs something equivalent, we will use yours.
PythonXGBoostscikit-learnLLM APIsRetrieval (RAG)PostgreSQL / pgvector
Questions
What clients ask first
Will you build a system that decides automatically about people?
No. Where a system judges people (hiring, eligibility, credit, ranking), it produces a recommendation with visible reasoning and a person decides. This is a firm line, and it is in our terms.
Do we need a large dataset?
Not always. Some problems are better served by retrieval over documents you already have. If your data genuinely cannot support a model, we will say so in the first week rather than the fourth month.
How do you stop the model quietly degrading?
The evaluation harness stays in the pipeline after launch, with the review overrides as an ongoing signal. Drift you cannot see is drift you cannot fix.
Related
Usually needed alongside
Data engineering & integration
Getting data out of the systems holding it hostage and into one place that can be queried and trusted, with schema…
See detailsWorkflow & process automation
The recurring manual work that consumes a week every month, rebuilt as durable jobs that log what they did, recover…
See detailsProduct engineering
Web applications, internal tools and APIs, built on the assumption that somebody else will maintain them one day,…
See detailsGet started
Is this the problem you have?
Describe it in three sentences and we will tell you what the first two weeks would look like.