Agent Gym

To improve an agent that negotiates recruiting, you want examples of recruiting negotiations going well. Those conversations happen in private inboxes and phone calls, they are commercially sensitive, and a platform accumulates them at the speed of real recruiting cycles — which is to say, far too slowly to train on.

So the environment manufactures them. A coach agent and an athlete agent talk to each other across three structured phases — match evaluation, deal negotiation, then enrollment and logistics — with terminal states at each boundary: no match, match confirmed, deal accepted, deal rejected, enrollment complete, stalled.

What keeps it from being fiction is that the participants are not invented. Both profiles are pulled from the platform's knowledge graph — real athletes with real event times, real coaches at real programs with real roster needs — alongside the NCAA rules that apply to that division. There is a generated fallback when the graph cannot supply a profile, but the default is grounded. One in five athletes is flagged international, which branches the episode into F-1 visas, I-20 issuance, and SEVIS logistics rather than skipping the hardest part of the process.

The agents are Claude. The judge, for automated training, is GPT-5. That is deliberate: a model grading its own output has an obvious conflict, and cross-model evaluation is the cheapest available defense against a system that learns to satisfy its own preferences rather than to negotiate well. An earlier heuristic judge was removed entirely — every episode is now LLM-scored.

Scoring is explicit rather than a single vibe rating. Negotiation turns are weighted: offer-probability delta at 50% — is this conversation actually moving toward resolution — then factual grounding at 20% for citing real times, thresholds, and deadlines, regulatory compliance at 20% for staying inside NCAA rules, and sentiment alignment at 10%.

Solo research turns use a different rubric entirely: prospect quality 35%, research depth 25%, NCAA compliance 20%, action readiness 20%. And the judge is told which phase the turn belongs to, so a discovery turn is scored for breadth while a planning turn is scored for whether anyone could act on it. The same output would be graded differently depending on when it arrived — which is the point.

Beyond the two-agent negotiation, two solo modes drop a single agent into the platform with real tools and let it work: Cypher queries against athletes, schools, and stats; a search over NCAA rule nodes; and vector search over past episodes.

The coach mode mimics what a coach actually does on the product — choose a research objective, run one targeted query, read the results, decide what matters and what to look at next. The athlete mode does the inverse: find programs by sport and division, benchmark its own times against athletes already at those schools, check eligibility and scholarship rules.

These modes exist because the negotiation simulation only exercises dialogue. The solo modes exercise the thing that makes the dialogue credible — whether an agent can find the right facts in a large graph before it opens its mouth.

Before an episode starts, the environment builds a query out of the sport, division, and both profiles, embeds it, and runs cosine similarity against every stored episode. Only episodes scoring at or above 0.6 total reward and ending in a genuinely good terminal state — deal accepted, enrollment complete, match confirmed — are eligible. The top three contribute their opening messages as few-shot examples, prepended to both agents' prompts.

The documentation is unusually specific about what this is not doing: it does not search mid-conversation, it does not splice in the highest-scoring individual message, and no humans are involved — it is agents learning from prior agents. What it provides is successful opening patterns, and the next step is named as mid-episode retrieval that finds the best continuation at the current phase.

The loop then leaves the gym. When a user's message to the platform's production assistant mentions recruiting, similar past episodes are pre-fetched from the same store and injected into its system prompt. Simulation is not a side experiment here; it is the mechanism by which the shipped agent gets better.

A human can steer a run in progress, and the design decision worth stating is what happens when they do: nothing pauses. The simulation streams turn by turn over SSE and never waits for input.

Feedback typed into the panel lands in an in-memory queue keyed by episode. At the start of the next turn the queue is drained, appended to the relevant agent's system prompt as a guidance block, and cleared. Submit something mid-generation and it applies to the following turn, not the current one.

That is asynchronous coaching rather than pause-and-resume, and it is the honest tradeoff: a simulation that blocks on human attention cannot also run unattended once an hour. Documenting it precisely matters more than the mechanism itself — the failure mode otherwise is someone typing guidance, watching the current turn ignore it, and concluding the feature is broken.

Cloud Scheduler triggers a training run every hour against a bearer-token-protected endpoint, with the secret in Cloud Secret Manager and the job itself provisioned through Cloud Build rather than clicked into a console.

It generates one episode per run, cut down from five. That number is a cost decision written into the README rather than discovered on a bill: every episode is two agents plus a judge across up to twenty turns, and the arithmetic on a system that runs unattended forever is worth doing before you turn it on rather than after.