Self-Improvement Loops: 2.3× Action Recall and 20% More Throughput

Over 6 days 16 hours, Codex operated a model-training research program for VoicePing’s meeting model. The work found an action-improving direction, established a reproducible current baseline, and documented the approaches that produced weaker results.

A separate inference study increased completed-request throughput by 20.2%, reduced mean complete-response latency by 18.1%, and raised the meeting-quality score from 73.42 to 76.40. First-pass format validity remained 22 of 25.

Human engineers defined the objective, evaluation rules, and adoption criteria before each workstream began. Codex handled the repeated research work inside those controls: changing code, launching GPU jobs, monitoring failures, scoring outputs, comparing candidates, and recording the next hypothesis.

This article explains the measured results, the experiments that shaped them, and the operating method behind both independent workstreams.

Results and impact

Both figures report results from the repeatable 25-meeting development evaluation. Production validation requires fresh examples.

Table of contents

• How we kept the research loop measurable
• Case study 1: six days of agent-operated model training
• What the training experiments changed and taught us
• Case study 2: optimizing inference with model weights fixed
• What improved across the two independent cases
• Rules for building a reliable agent research loop

How we kept the research loop measurable

Each measured result guided Codex’s next bounded change. Depending on the workstream, Codex could work on code, data selection, training recipes, prompts, or serving settings. Human-controlled evaluation decided whether to retain each candidate.

We divided the work this way:

We kept the evaluation under human control. We stored the evaluation data, scoring rubric, and required behaviors outside the files and settings Codex could change. Codex received the results and used them to improve the candidate. This kept every score tied to the same definition of success.

The two VoicePing cases had different goals:

Each case had its own objective, experiment history, and allowed changes. They shared the same evaluation-controlled method.

Case study 1: six days of agent-operated model training

The main research objective was to improve a Gemma 4 E4B meeting model that produces a title, summary, action items, and topics. Human engineers clarified the objective before the first long run. Every output area had a minimum quality requirement. The evaluation also tracked missed actions and invented actions as separate errors.

The Codex task history ran from July 27 to August 3, 2026: 6 days 16 hours elapsed. During that period, Codex worked through many research stages:

• defining and correcting the evaluation contract
• changing training and evaluation code
• running many separate Group Relative Policy Optimization (GRPO) and supervised fine-tuning jobs
• generating meeting outputs from saved checkpoints
• scoring those outputs with a separately configured Qwen evaluator
• checking repeatability, seeds, model lineage, and runtime identity
• diagnosing failed runs and preparing reports
• monitoring long jobs and waiting on shared compute or evaluation services

Training produced candidate checkpoints. Evaluation, diagnosis, and recorded decisions turned those checkpoints into useful research evidence.

Figure 3. Humans fixed the objective and evaluation. Codex operated the experiment and prepared each result for review.

We defined the evaluation before training

We used a repeatable evaluation set of 25 real meeting transcripts: 20 typical meetings and 5 long meetings. Each checkpoint generated one output per meeting and received separate judgments for summary, action items, topics, title, structure, and long-meeting behavior.

The set supported fast, repeated decisions and covered several product behaviors. We used it as a development benchmark because the same meetings guided many experiment choices.

We kept the Qwen-based evaluator consistent across candidates by pinning its prompt, model revision, parsing logic, and generation settings. Repeatability checks measured scoring stability, and every record included the scorer and training-code versions.

How we selected candidate checkpoints

The overall score summarized performance across the evaluation set. Separate checks measured the product requirements that needed their own thresholds, including action coverage and invented actions.

Every candidate had to meet these required checks:

• summary, action, topic, and title quality
• typical-meeting and long-meeting performance
• valid first-pass output structure
• action emission on the 16 meetings that contained real actions
• zero invented actions on the 9 no-action meetings
• fixed evaluation membership, one-attempt generation, and pinned scorer identity

We adopted a new baseline only after it improved the target behavior and met every required check.

What the training experiments changed and taught us

The six-day loop explored several directions. The table below replaces internal run names with the question each experiment asked, the measured outcome, and the resulting decision.

This summary replaces internal run IDs with the question and learning from each experiment. The training program found an action-improving direction, ended several approaches that produced poorer results, established a reproducible current baseline, and applied the required product checks during selection.

Live reward monitored training health

During training, the reward signal answered operational questions: were evaluator calls returning, were groups producing useful variation, were gradients active, and were values clipping or becoming unstable?

Checkpoint quality came from the separate meeting evaluation. Across 14 runs with comparable telemetry, its descriptive Pearson correlation with mean live reward was −0.315. In one example, a run reached a live mean of 0.8926 and a meeting-evaluation score of 74.5375.

Figure 4. Live reward monitored run health; the independent meeting evaluation decided whether quality improved.

The two measurements covered different populations and generation conditions. Live reward averaged sampled candidates on training prompts. The meeting evaluation scored one deterministic output for each meeting in the development set. Batch difficulty also changed with action density, context length, and evidence richness. We used the reward curve for run health and the meeting evaluation for checkpoint selection.

Failed runs needed different labels

The agent also improved how it interpreted failures. We used four reader-facing outcome types:

• Completed experiment, negative result: training and evaluation finished, and the candidate scored below the requirements. This is valid evidence about the idea.
• Execution failure: training or scoring stopped before the tested idea could be judged. Fix the runtime, then rerun the same experiment.
• Invalid comparison: the runtime, evaluator, seed, or artifact lineage changed enough to alter the meaning of the score.
• Provisional result: monitoring data exists, and the final checkpoint or independent evaluation is still in progress.

For example, one fine-tuning attempt failed at optimizer step 0 because two installed libraries disagreed about the form of model logits. Training never began, so we classified the result as an execution failure. After correcting the runtime, the rerun used the same recipe and data to preserve the original research question.

The experiment history improved each new decision

Every experiment wrote its configuration, source revisions, seeds, artifact hashes, aggregate results, decision, and next hypothesis to an append-only experiment log. Negative results remained searchable.

That history changed later behavior. Before suggesting another curriculum-only run, Codex could see the previous result for that approach. Before treating selected high-scoring samples as evidence for fine-tuning, it found the earlier transfer failure. The loop became less likely to repeat a discarded approach and better at separating an experiment result from a runtime problem.

This experiment memory improved the reliability and information value of every later research decision. It became another part of the self-improving loop alongside changes to model weights.

Limitations of the training self-improvement loop

The training loop chose its next change from evaluation results and experiment history. This creates four limits:

• The loop follows the goal humans define. Anything missing from the rubric, such as tone, factual nuance, or downstream usefulness, cannot guide the next experiment.
• Evaluator bias can repeat. The same Qwen judge helped select checkpoints and shape the next hypothesis. One blind spot could therefore affect several experiments. Regular expert review or a second evaluator is needed.
• Memory can preserve mistakes. A bad comparison or incomplete diagnosis can affect future choices. Records need exact artifact and runtime details, correction status, and human review.
• Humans still control scope and release. Codex could change the data mix, rewards, schedules, and training code. Humans approved new data, changed the objective, accepted quality risks, and decided whether to deploy. They must step in when the loop stalls or finds an unclear tradeoff.

Case study 2: optimizing inference with model weights fixed

The second case had a simpler and independent objective: find the best serving configuration for a given Gemma 4 E4B model and VoicePing’s meeting workload. Model weights stayed fixed. We accepted a candidate after it improved complete-response speed or memory use and met the defined quality checks.

Start from the workload the product actually sends

The same 25-meeting collection provided a realistic request shape for this independent study. Inputs reached 27,779 prompt tokens and 31,779 total tokens after reserving room for the response, so the serving profile needed a 32K context window. VoicePing returns a complete meeting summary in one response, so we measured full-response latency.

We tested four concurrent requests and compared the base decoder with several multi-token prediction depths. Depth 4 was the best measured setting:

• Completed requests per second increased from 0.690 to 0.829: +20.2%.
• Mean complete-response latency fell from 5.32 to 4.35 seconds: −18.1%.
• 95th-percentile (P95) complete-response latency fell from 10.47 to 9.12 seconds: −12.9%.
• The meeting-quality score moved from 73.42 to 76.40. First-pass format validity remained 22 of 25.

Depth 1 added assistant-model overhead and reduced performance. Deeper settings drafted more tokens, and their later token positions had lower acceptance. Depth 4 delivered the highest end-to-end throughput for this workload. The vLLM multi-token prediction documentation describes the mechanism.

Results from the other serving configurations

• Prefix caching reused 2.98% of prompt tokens. Meeting-specific information appeared before the shared instructions, leaving a small reusable prefix. Prompt reordering became the next useful change to test. The vLLM prefix-caching documentation explains the requirement for exact reusable prefixes.
• Alternative attention and sampling paths kept the existing result unchanged. One was incompatible with the model’s head dimensions. Another matched native throughput and required an extra deployment component.
• Quantization labels gave an incomplete picture of memory use. A standard 4-bit path reached about 11.7 GB. The official mixed-bit mobile checkpoint reached a measured 6,708 MiB peak. The embedding and output layers explained much of the difference.
• The llama.cpp path used less memory. In the same one-request comparison, vLLM completed the set faster. The llama.cpp path produced 9 of 25 first-pass-valid outputs. We selected vLLM for the current serving profile; the remaining llama.cpp semantic check was still in progress at the evidence cutoff.

The inference case measured the complete serving system across latency, throughput, memory, format validity, and output quality. OpenAI describes a broader version of this approach in its account of GPT-5.6 inference efficiency , where routing, scheduling, kernels, caching, and model implementation are measured as one serving stack.

For more background on the product workload, see VoicePing’s automatic meeting transcription guide and our production speech-model engineering case study .

Limitations of the inference self-improvement loop

The inference loop chose serving settings from benchmark results. This creates three limits:

• The loop optimizes the workload humans define. The selected configuration used 32K meeting transcripts and four concurrent requests. Short requests, burst traffic, another GPU, or another model need their own evaluation loop.
• Unmeasured costs remain invisible. Startup time, deployment complexity, recovery behavior, infrastructure cost, and observability have no effect until humans add them to the acceptance criteria.
• Live changes require safety controls. Continuous production tuning needs canary traffic, rollback rules, drift monitoring, an external quality check, and explicit approval. This case evaluated configuration changes offline.

What improved across the two independent cases

In our work, “self-improvement” covered three editable layers:

The training and inference projects remained independent. Both used the same operating method: clarify the objective, fix the definition of success, give the agent a defined area to change, and retain results that pass the evaluation.

Rules for building a reliable agent research loop

Use this checklist before assigning an agent a long-running optimization goal.

The operating sequence is short:

1. Clarify the objective with the human owner.
2. Freeze the evaluation data, scoring rules, and required behaviors.
3. Give the agent one editable surface and a bounded budget.
4. Run and monitor the experiment, then save the candidate with its artifact hash.
5. Evaluate under the unchanged test.
6. Keep, discard, or diagnose the result.
7. Record the learning before choosing the next change.

This resembles the edit-run-measure loop in Andrej Karpathy’s Autoresearch , extended with separate component checks, evaluator versioning, private-data boundaries, and human adoption authority.

Conclusion

Our self-improving AI research loops used a clear division of responsibility. Humans chose the product objective and evaluation. Codex handled the repeated research work: changing one mechanism, running it, measuring it, diagnosing failures, and preserving the result.

The main training case ran for 6 days 16 hours, tested many directions, rejected candidates that failed required behaviors, and established a reproducible baseline. The separate inference case found a configuration with 20.2% higher completed-request throughput and 18.1% lower mean response time. Its quality score increased from 73.42 to 76.40, and format validity remained 22 of 25.

The transferable method has three parts: define success before the run, keep evaluation under human control, and use measured results to decide what the agent tries next.

References

1. OpenAI, How GPT-5.6 fuses frontier intelligence with frontier efficiency , July 29, 2026.
2. Andrej Karpathy, Autoresearch .
3. vLLM, Multi-token prediction documentation .
4. Hugging Face, TRL GRPO Trainer documentation .
5. vLLM, Automatic prefix caching documentation .
6. ggml-org, llama.cpp .