Searching Through a Virtual World

At Whitney AI, we did a deep dive on what it would take to build an enterprise search application of our own by doing what we know best: post-training. We trained a model that retains 91.1% of frontier search performance with 600x fewer total parameters — small enough to easily run on-prem, or even locally on a single machine.

Within enterprise search, we tackled the deceptively complex task of building a daily brief agent.

Its principal responsibilities are, for any given day and employee in an organization:

There’s a fair amount of information the model needs to know in order to get this right. For example:

None of the above points are easily enumerable in a system prompt. But the problem is verifiable, meaning the model can learn from experience by getting enough reps in, and that makes it amenable to training.

We also wanted performance as close to the frontier on our task as possible and a model compact enough for local usage. Local deployments drive inference spend down to the marginal cost of the hardware a company already owns. These were aggressive goals, and we have found that well-designed post-training unlocks the greatest gains over pure context or harness optimization, so it’s where we started.

The first step of building our daily brief agent is to start a company. No, really — we need high-fidelity corporate collaboration data for the agent to learn how to reason about action items and surface them in briefs. Training directly on a real company’s enterprise data is usually impossible, and where it isn’t, the legal and privacy work would fill a longer article than this one. So we synthesize a company ourselves.

We start with source extraction: we choose a real target organization and extract current and recent project initiatives from their online newsroom and blog posts, paired with results from Exa API’s web and news search. This nets us a realistic set of initiatives on which to ground our generation, which we refine through deduplication and filtering into a canonical project catalogue.

One project in the catalogue aims to digitize contractors’ material lists. We’ll use its staff and timelines as examples in the next two sections.

Next we staff the projects by binding roles to them. A company policy change might need corporate counsel, HR, and a decision-making executive involved. A product feature A/B test flight might need designers, engineers, a product manager, and so on. Given a complete staff list, we generate employee identities as needed. Support staff are shared between projects up to a reasonable bandwidth limit — we cap at 10 simultaneous projects. Identities include both names and employee credentials such as email addresses.

Ultimately, we need unambiguously verifiable brief items, so we ensure these are available on construction by generating project timelines with clear milestones first. Given a project milestone, we then rotate through some subset of the staffed employees as the focal identity and populate their personal timelines with the dated action items needed from them for completing the overall project.

Given an employee’s personal timeline, we now need their Slack, email, and other corporate activity data that the model must subsequently search through. Not all generated content is relevant to the project, just like what happens in real life. Activity content needs to be generated such that, in aggregate, each focal employee’s respective deadlines are communicated clearly in advance and therefore made discoverable to the model.

We did not want to surface every single miscellaneous to-do item in the preceding days as reminders, so we classified which deadlines were high priority and only those were eligible for one- and two-day prior reminder alerts. The rest were meant for day-of notice only.

The second step of building our daily brief agent is to create an RL environment, or a virtual world designed for our task. Our agent needs some stateful execution substrate over which it can search through our corporate data and generate briefs. We use Vercel’s emulate package to stand up full-parity API clones of Slack, Gmail, Google Drive, and Calendar and expose the read-only operations as tools to the model. We fork the emulator and add a virtual clock. Events are ingested in batches with timestamps and replay as the clock advances, and the world can be reset to any point in time. Without this machinery, a brief written for a given day would be generated against a world that already knows how the week turned out, contaminating our results.

With the RL environment in place, our harness’s tooling layer is fixed: 19 functions from our four service clones that together enable exhaustive search across our corporate tenant. We perform lightweight harness optimization across tool descriptions and prompts thereafter, ensuring that, among other things:

The model could have figured these out independently given enough training — but we find co-evolving the harness and model together leads to even greater improvements than doing either alone.

After iteratively generating, refining, and regenerating our data until it passes human review for quality and realism, we sanity-check the dataset by evaluating off-the-shelf models on it. Employees, projects, and timelines are kept entirely disjoint across train and eval sets to prevent contamination.

We run Qwen3.8-2.4T-A95B on our 200-example evaluation set, observing a failure-adjusted macro F1 score of 0.698. The challenge of this task lies in search: can we find the right points of evidence that reveal an actionable, incomplete deadline owned by the focal employee? Once the citation set is determined, the actual text generation of the user-friendly brief is trivial — so we grade on the citation set directly. With this context, we can define our metric:

• F1: Our balance of recall and precision over cited evidence compared to the golden set for each user on each day.
• Macro averaging: Done across focal employees. Some have busier workloads or more demanding projects than others and simply have more items due.
• Failure-adjusted: Accounts for examples where the model cannot generate a citation set at all, often because it ran out of tokens or turns. They receive a score of 0. Qwen3.8-2.4T-A95B had a completion rate of 96%, meaning 192 of our 200 examples produced a correctly formatted brief¹, irrespective of citation quality.

We use Qwen models as both our frontier target (Qwen3.8-2.4T-A95B) and our base model to train (Qwen3.5-4B); all data is generated using a variety of OpenAI models, which partially mitigates self-preference bias concerns.

Ensembling several models like Qwen3.8-2.4T-A95B, gpt-5.6-luna, and gpt-5.6-sol together across our task raises the failure-adjusted macro F1 to above 0.90. Some of the models share a family with the models used to generate the data, so we read this as corroborating rather than independent evidence, but the combined score further convinced us of our dataset’s integrity. Ensembling is far more expensive to serve, so we treat the top-of-the-line Qwen3.8 result above as the goal to meet in this work, with outperforming the ensemble approach left for future iterations.

With our goal clear, data ready, and metrics defined, we proceed by measuring the performance of our base model. Qwen3.5-4B scores less than half of Qwen3.8-2.4T-A95B with a failure-adjusted macro F1 of 0.29. This is partially due to a much lower completion rate of 66.5% — with the model commonly getting lost in search until it runs out of budget. But even its unadjusted macro F1 score, the mean over completed episodes only, is 0.436, so we have work to do in driving up both completion rate and search rigor.

We additionally baseline the mid-sized Qwen3.8-27B model. The 2.4T model is slower to iterate with, and the more data we have about same-family models, the more options we have for dense signal transfer through strategies like on-policy distillation (OPD). The 27B model is particularly promising. It scores a failure-adjusted macro F1 of 0.664, with a completion rate of 79%. Its reasoning tends to run long and exhaust token limits, but when results do complete, it tends to be even stronger than Qwen3.8-2.4T-A95B with an unadjusted macro F1 of 0.841.

Our first instinct is to run various forms of OPD. Qwen models versioned from 3.5 to 3.8 share a vocabulary which makes dense token-level distillation possible. Through OPD we directly align the logprobs of what the student thought to generate and how the teacher instead scored those tokens, sometimes even over top-K ranges which transfers the teacher’s soft knowledge instead of just its final decision. Common industry practice prefers this over alternatives like sequence-level supervised fine-tuning (SFT) due to its faster convergence, better performance transfer, and increased robustness.

But common wisdom ends up getting overturned here. Running OPD from 27B to 4B sees the brief completion rate quickly approach 0%; the student model’s reasoning length balloons and maxes out the token cap before the closing </think> tag is reached. This is not an unprecedented phenomenon, with several recent papers observing and mitigating runaway lengths during OPD²³.

Despite precedent, one still wonders: the teacher had a 79% completion rate — why is our trained model doing so poorly? The answer lies in the student state distribution over the course of training. The teacher generates well from its own coherent states, but during OPD we ask it to supervise increasingly repetitive student prefixes such as, “Actually ..., wait ..., let me double check ...” From those prefixes the teacher assigns low probability to </think>, because stopping there would in fact be wrong. This negative advantage turns into gradient pressure that actively pushes the student away from the stop token, producing more garbled prefixes and lowering stop-token probability further still.

In the limit, a student that genuinely converged to the teacher would generate the same coherent states the teacher is familiar with and stop cleanly; our objective is not misstated. But the optimization path to that objective runs through a regime where teacher supervision is length-inflating, and the loop is self-reinforcing enough that training never leaves it.

Even the teacher struggles to leave those bad student states. We validate this by sampling random student trajectories, truncating them at different lengths, 50%, 75%, and 90% of the maximum token limit, and measuring the teacher’s ability to recover them to a structurally correct brief.

When the teacher does succeed — which only happens roughly 40% to 50% of the time given a fixed recovery budget — it can discard the student reasoning entirely and start from scratch, like the example above. Here, the model closes the </think> tag after the Sunday deduction and emits a valid tool call in 1,295 tokens. It still rederives the same date arithmetic the student was stuck on, just without looping.

We apply the suggested mitigations from the literature: stop-token clamping to suppress negative advantages from the teacher that would encourage runaway lengths, a counterbalancing reference KL to minimize overall drift from the original student policy, and an overlong penalty⁴ discouraging length itself before the cap is ever reached. None yield success. Reverse KL is measured under the student’s own state distribution; it can fall steadily with the student better matching the teacher, pointwise, on the states it visits — while the state distribution it is visiting degrades.

So we swap out on-policy methods in favor of off-policy ones, specifically dense supervised fine-tuning, to build a better prior into the model. Our teacher model is the same as before, Qwen3.8-27B. We measure forward KL instead of reverse KL because we are now training on teacher states rather than student states; this metric is more immune to student state-distribution deterioration.

We trade for another failure mode instead: behavioral cloning gives us clean supervision but no coverage of the states the student will actually visit at inference, so distribution mismatch and compounding errors return. We accept that for now. The point is a prior good enough that a later on-policy stage has something to recover from.

We evaluate the final checkpoint from Figure 2. Its unadjusted macro F1 is 0.702, well above the base model’s 0.436 — off-policy training bought us most of the search quality we needed. But completion rate sits at a low 56%, which drops the adjusted macro F1 to 0.394, only 0.104 better than baseline. The capability is now in the model; we just need to make it reliably finish episodes to show it.

We take this run’s checkpoint as the starting model for recovery-based RL, aiming to hold unadjusted F1 where it is and repair the completion rate around it. On rewards:

We evaluate the final checkpoint and observe a failure-adjusted macro F1 of 0.636 with a 92% completion rate. Compared to the distillation checkpoint, unadjusted macro F1 after RL moves from 0.702 to 0.691 while completion climbs from 56% to 92%. This is not a quality regression. At 56%, the completed episodes are a selected set, the ones the model found tractable. At 92%, we are scoring a substantially harder mix that includes everything which previously ran out of budget.