Start with a clear task
Requirements and project context give the agents a concrete brief. The scheduler coordinates the work that follows.
The brief
Code written is only the beginning.
An AI agent can finish an implementation while leaving gaps in behaviour, integration or documentation. Letting the same agent decide that its own work is complete makes those gaps harder to catch.
We built tPanel AI QA system as an internal platform to coordinate development, give verification its own roles and retain an accurate record of the delivered work. The central goal is better delivery quality.
THE DELIVERY PROCESSHuman decisions throughout
IntentDefine the work
Requirements and project context
CoordinationBuild
Implement the work
Quality checksVerify
- Review 1
- Testing
- Review 2
- QA
KnowledgeRecord the result
Final code, findings and documentation
Blocking finding -> Repair -> Re-checkProduction release stays with a human
Keep release decisions with people
People resolve escalations and decide when to release. Independent AI roles carry out the review and QA stages.
AI-generated workplace illustrations, not photographs of tPanel staff or premises.
- Built for
- tPanel's internal delivery workflow
- Core design
- Code coordinates. Agents execute.
- Release control
- Human approval and deployment
Independent verification
Different stages. Different questions.
Four independent verification stages check changes before they reach a human. Each stage has a distinct job. Review 2 runs only when a task spans more than one module.
Code
AI
Four verification stages
01Review 1 (Module)
- Run lint, formatter and type checks against a baseline.
- Fast, deep and lead reviewers assess the module diff.
- Return only new, in-scope P0/P1 issues.
Pass
02Testing
- Run the full configured test suite.
- A dedicated tester agent adds missing tests.
- Re-run failures once and classify them before returning work.
Pass
03Review 2 (Integration)
- Merge module branches and check shared contracts, validation and trust boundaries.
- Run the full suite on merged code.
- Multi-module tasks only.
Pass
04QA
- Start the real app and follow the QA script.
- Check user workflows against the original requirements.
- Report pass or fail with reproduction steps.
Human final gate
- 01 / Review 1 (Module)
Is each module's change correct and in scope?
Lint and type checks, then three independent reviewers inspect each module's diff.
- Runs the project's lint, formatter and type-check commands. It takes a baseline on the starting commit, so only new problems count.
- A fast reviewer reads the whole diff and lists candidate issues.
- A deep reviewer re-checks only the high-risk or uncertain candidates, and confirms or rejects each one with evidence.
- A review lead merges both into one report and rates each finding P0, P1 or P2.
- Only P0/P1 issues that this change introduced, within the task's requirements, send work back. P2 issues are recorded as deferred items.
- From the second round on, only the new code is reviewed.
- Reviewers can't change code. The working tree is compared before and after each review.
- 02 / Testing
Does it actually work?
Run the full test suite. A dedicated agent adds the tests that are missing.
- Runs the project's configured test suites.
- A tester agent reads the diff and adds the minimum tests the change needs.
- Failing tests are re-run once, which filters out flaky results.
- Each failure is sorted into one of five types: product bug, test bug, environment, pre-existing or flaky. Only product bugs introduced by the change go back.
- Known failures pass only if a human approved adding them to the baseline.
- Test runs are isolated and serialised, so parallel tasks can't interfere.
- 03 / Review 2 (Integration)
Do the parts fit together?
Merge all module branches and check how the parts fit together.
- Merges every module branch (backend, frontend, etc.) into one integration tree.
- Checks the modules against each other: shared contracts, consistent validation and trust boundaries.
- Runs the full test suite again on the merged code.
- Runs only when a task spans more than one module.
- 04 / QA
Would a real user get what was asked for?
Run the app and verify user workflows against the original requirements.
- Starts the real application, following the project's QA script.
- Works through user workflows from the user's point of view.
- Checks the result against the original requirements, not against the code.
- Reports pass or fail with steps to reproduce.
The scheduler
Make the process dependable.
The early pipeline relied on an AI team leader following a written procedure. It could skip steps, apply gates inconsistently or continue re-planning without reaching a clear end state.
Orchestration moved into a scheduler written in code. Agents still perform the work within each stage; explicit state transitions, checkpoints and rework rules control how the task moves forward.
A recoverable workflow
- Plan + buildWork from the brief and save stage checkpoints.
- Coded schedulerKeep one active run per task and classify interruptions.
- Continue safelyResume saved work, retry or pause.
Recover interrupted work
Stage checkpoints preserve completed work. A per-task lock prevents duplicate runs, so recovery can continue from a known state.
Interruption -> Checkpoint -> ResumeRespond to the right failure
Service errors, usage limits and product defects need different responses. A service interruption should not be recorded as a failed code review.
Classify -> Retry, pause or repairGive rework an endpoint
Scope and severity determine what blocks progress. Rework limits move unresolved decisions to a human instead of allowing an indefinite loop.
Check -> Bounded rework -> EscalateThese mechanisms support a more controlled process. Fewer sessions alone would not demonstrate higher software quality.
Project knowledge
Document what was actually built.
A useful project record explains the implementation, the decisions behind it and the issues that remain. It should reflect the final code, rather than repeat the original plan after the implementation has changed.
tPanel AI QA system connects task outcomes with project documentation, giving future work a clearer starting point.
Read about agent memory governance ->Delivery record
Final code + decisions + findings
The basis for documentation and future project context.
In development
Retrieval-augmented generation
We are adding RAG to help agents retrieve relevant project knowledge as they work. Its impact has not yet been evaluated.
From delivery to knowledge
- Delivered workFinal code, decisions, findings and task outcomes.
- Project documentationKeep an accurate record of what was built.
- Planned retrievalRAG is in development. Its impact has not been evaluated.
Under the hood
A shared platform with separate responsibilities.
- Agent runtime
- Claude Agent SDK and OpenAI Codex, with provider and model choices per role.
- Orchestration
- Python and asyncio, stage checkpoints, task locks and event journals for replay testing.
- Control plane
- FastAPI, PostgreSQL and a React interface for task records, project documents and quality settings.
- Project workspace
- Docker isolation, Slack conversations and MCP access to shared tools.
FAQ
How responsibility works
Quality checks, escalation and release decisions.
Is QA carried out by people or AI agents?
Code review and QA are carried out by separate AI agent roles. That separation provides independent checking from the developer, but it is not a claim of human code review. People make product decisions, handle escalations and control production deployment.
What happens when a check finds a problem?
Findings are assessed against severity, scope and the original requirements. Blocking issues return to development for repair and re-verification. Non-blocking findings are recorded in the outcome report. Rework limits prevent an unresolved task from cycling indefinitely; a human receives the remaining decision.
Does the platform deploy software automatically?
No. Agents can develop, test and merge work to the main branch. Production deployment remains a human decision.
Is RAG already part of the platform?
A retrieval-augmented generation (RAG) layer is being added to help agents retrieve relevant project knowledge. It is under development, and we are not claiming measured results from it yet.
Talk through your AI workflow
Tell us who uses it, what you use today and where the work gets difficult. We can discuss whether to adapt your existing tools or build something around your process.
Related work