Research Monograph Series · Parts I–III · Five CERN Zenodo Preprints

Single-Prefill Calibrated System-One Decision Models as Unified Process, Outcome, and Tree-Pruning Judges for Reinforcement Learning

从连续物理控制（MuJoCo v5）到单基座双适配器大模型 GRPO、多步代码与工具智能体、零额外基座开销自演进裁判及混合线性注意力前缀树定理证明的统一理论与实验体系

Author

Ziyang Zhang (张子阳)
Independent AI Research

Shared Architecture

Qwen/Qwen3.5-4B + Kev-4B
8.06 GB VRAM (-50.2% vs. Dual-Model)

Single-Prefill Latency

52.14 ms / rollout · 14.80 ms / tree node
0.012 ms zero-copy LoRA switch

Archival Corpus

5 English Preprints (59 pp.)
5 Chinese Editions (38 pp.) · 17 Benchmarks

GitHub Release v3.0.0 (All 10 PDFs) Source Code Repository Hugging Face Interactive Workbench Hugging Face Model & Benchmark Hub Kev-4B Technical Discussion

Synopsis

Reinforcement learning across both continuous physical control and large language model (LLM) post-training faces a fundamental trilemma between brittle binary outcome verifiers (which suffer from zero-variance group dead-zones $\sigma_G = 0$ and process-blind shortcut or test-tampering exploits), un-gated scalar process reward models (which succumb to additive Goodhart over-optimization), and autoregressive LLM-as-a-Judge decoding (which doubles GPU memory and incurs $6\times$ latency). Across this five-part monograph series, we resolve this trilemma by exploiting our architectural finding that TypeSafe AI's open-weight System-One decision model (jaredpalmer/kev-4b) is natively constructed as a rank-16 LoRA adapter plus a Layer-28 pointer extraction head on top of Qwen/Qwen3.5-4B. By co-locating the trainable policy adapter and the System-One judge adapter on a single shared Qwen/Qwen3.5-4B backbone in GPU memory, our framework reduces static and adapter VRAM by 50.2% (8.06 GB vs. 16.20 GB on an NVIDIA RTX PRO 6000 Blackwell 96 GB GPU) and extracts calibrated multi-rubric Score expectations alongside a multiplicative Noul validity gate in a single non-autoregressive prefill pass (52.14 ms).

01 Five-Paper Archival Index & Permanent CERN Zenodo DOIs

Each installment in the monograph series isolates a distinct structural bottleneck in modern reinforcement learning, accompanied by a full-length English preprint (11–12 pages), a full-length Chinese edition (7–8 pages), strictly controlled same-backbone (seed=42) verification experiments, and a permanent CERN Zenodo DOI.

Table 1. Summary of the five-paper System-One Reinforcement Learning Monograph Series, archival DOIs, evaluated benchmarks, and headline controlled empirical findings.

Monograph	System Designation	Evaluated Benchmarks (17 Total)	Primary Theoretical & System Contribution	Headline Metric	Official CERN Zenodo DOI	Manuscripts
Part I	`DC-Jev-RL`	5 MuJoCo v5 Tasks Hopper, HalfCheetah, Walker2d, Swimmer, Reacher	Single-prefill `Score + Noul` gating eliminates Hopper $v_x=0$ alive-bonus hacking ($-84\%$, $213.2\text{ ms}$ on L4).	2.471x ONS 100% (5/5) Win	`10.5281/zenodo.23439711`	EN 11p CN 7p
Part II	`DC-Jev-GRPO`	GSM8K, MATH-500, Countdown-24, IFEval-Strict	Single-backbone dual-adapter co-location ($-50.2\%$ VRAM); eliminates $\sigma_G=0$ dead-zones ($-89.6\%$) & yap-loops.	85.90% Mean +12.10% vs RLVR	`10.5281/zenodo.23441431`	EN 12p CN 8p
Part III-A	`Agentic-Jev-GRPO`	BFCL-v3 Multi-Turn, MBPP+, HumanEval+, SWE-bench-Mini	Turn-level telescoping PBRS with $84.6\%$ KV prefix reuse ($52.14\text{ ms/turn}$); trajectory-min `Noul` blocks test tampering.	78.23% Mean 48.6% SWE-bench	`10.5281/zenodo.23442087`	EN 12p CN 8p
Part III-B	`CoEvo-Jev-GRPO`	MATH-500, AIME 2024/2025, AMC-12, OlympiadBench	Verifier-anchored bilevel co-evolution of Layer-28 Pointer Head ($+0.02\text{ GB}$ VRAM, $0.84\text{ ms}$) contracts ECE $0.048 \to 0.011$.	59.28% Mean 36.7% AIME (+16.7%)	`10.5281/zenodo.23442137`	EN 12p CN 8p
Part III-C	`Tree-Jev-GRPO`	MiniF2F-Lean4, ProofNet, MATH-500 L5, GPQA-Diamond	Gated-DeltaNet prefix-tree KV sharing ($3.81\times$ compression) + $14.80\text{ ms/node}$ speculative `Noul` pruning ($-66\%$ FLOPs).	60.50% Mean 64.8% MiniF2F	`10.5281/zenodo.23442169`	EN 12p CN 7p

02 Unified Single-Backbone System-One Formulation

Let $W_{\text{base}} \in \mathbb{R}^{L \times D}$ denote the shared bfloat16 parameters of Qwen/Qwen3.5-4B ($7.94\text{ GB}$). Both the trainable policy adapter $\theta_{\text{pol}}$ and the System-One judge adapter $\theta_{\text{kev}} \cup \{W_{\text{ptr}}^{(28)}\}$ are mounted on $W_{\text{base}}$. In a single non-autoregressive prefill forward pass over prompt-rollout pair $(q, o_g)$, Layer-28 pointer attention extracts calibrated rubric probabilities $(p_{\text{proc}}, p_{\text{inst}}, p_{\text{conc}}) \in [0, 1]^3$, epistemic confidence weights $w_i = 1 - \sigma_i$, and the null-calibrated validity probability $p_{\text{noul}} = 1 - p_{\text{null}}$:

$$g_{\text{noul}}(q, o_g) = \text{clip}\!\left(\frac{p_{\text{noul}}(q, o_g) \cdot \bigl(0.20 + 0.80 \, p_{\text{proc}}(q, o_g)\bigr)}{\tau_{\text{noul}}}, \; \epsilon_{\text{floor}}, \; 1.0\right),$$ $$R_{\text{total}}(q, o_g; k) = g_{\text{noul}}(q, o_g) \cdot \Bigl[\alpha(k)\, R_{\text{ver}}(q, o_g) + \beta(k)\, w_{\text{proc}} p_{\text{proc}} + \delta(k)\, w_{\text{inst}} p_{\text{inst}} + \eta(k)\, w_{\text{conc}} p_{\text{conc}}\Bigr] + \lambda_{\text{pbrs}} \Phi_{\text{dens}}(o_g).$$

Theorem 1 (Elimination of Zero-Variance GRPO Dead-Zones & Negative Hack Advantage).

On any frontier prompt $q$ where all $G$ rollouts fail the binary outcome check ($R_{\text{ver}}(q, o_1) = \dots = R_{\text{ver}}(q, o_G) = 0$), binary RLVR yields $\text{Var}_G(R_{\text{ver}}) = 0$ and vanishing gradient $\nabla_{\theta} \mathcal{J}_{\text{GRPO}} = 0$. Because $p_{\text{proc}}, p_{\text{inst}}, p_{\text{conc}}$ are real-analytic softmax expectations over Layer-28 hidden states $H^{(28)}(q, o_g)$, their level sets have Lebesgue measure zero, guaranteeing $\text{Var}_G(R_{\text{total}}) > 0$ almost surely. Moreover, for any mixed group containing a valid proof $o^*$ ($g_{\text{noul}} = 1$) and a shortcut, yap-loop, or test-tampering rollout $o_{\text{hack}}$ ($g_{\text{noul}} \le 0.27$), the standardized GRPO advantage satisfies $A_G(o_{\text{hack}}) < 0 < A_G(o^*)$.

03 Interactive System-One (`/v1/systemone`) Calibration Instrument

Inspect the live Layer-28 pointer telemetry measured on our NVIDIA RTX PRO 6000 Blackwell (96 GB) across seven canonical rollout regimes from Parts II and III, or perturb the rubric probabilities below to verify the exact response of the multiplicative Noul gate $g_{\text{noul}}$ and composite reward $R_{\text{total}}$.

Instrument: Layer-28 Pointer Head & Multiplicative Gate Evaluator

Backbone: Qwen/Qwen3.5-4B · Judge: jaredpalmer/kev-4b · Prefill: 52.14 ms

Binary Verifier $R_{\text{ver}} \in \{0, 1\}$1.00

Logical / Step Process $p_{\text{proc}}$0.9258

Constraint / Schema Precision $p_{\text{inst}}$0.8914

Reasoning Density $p_{\text{conc}}$0.9445

Non-Null Validity $p_{\text{noul}} = 1 - p_{\text{null}}$0.6288

Curriculum Horizon $k / K \in [0, 1]$0.65

Prompt [MATH-500]: Solve 3^(2x-1) = 27^(x-1) and place the final value in \boxed{}. Rollout (58 tokens): Rewrite 27 as 3^3 so 3^(2x-1) = 3^(3x-3). Equating exponents gives 2x - 1 = 3x - 3, hence x = 2. \boxed{2}

Reward / Gate Signal	Formulation	Value
Binary RLVR	`R_ver`	1.0000
Un-Gated Linear PRM	`0.55 R_v + 0.30 p_p + 0.15 p_i`	0.9568
Multiplicative Noul Gate	`g_noul(p_noul, p_proc)`	1.0000
DC-Jev-GRPO (Ours)	`g_noul * Sum(w_i p_i)`	0.9522

Diagnostic Analysis: Full validity gate activation (g_noul = 1.0000). Rigorous 58-token derivation receives R_total = 0.9522 and positive standardized GRPO group advantage ($A_g > 0$).

04 Strictly Controlled Same-Backbone Verification (`Qwen/Qwen3.5-4B`, `seed=42`)

To eliminate any confounding effect from foundation-model capacity differences, all competing reward paradigms in Table 2 are initialized from the bit-for-bit identical Qwen/Qwen3.5-4B base weights and identical initial rank-16 LoRA tensors (seed=42, lora_alpha=32), trained via online PyTorch GRPO backpropagation on our NVIDIA RTX PRO 6000 Blackwell (96 GB).

Table 2. Controlled Same-Backbone (Qwen/Qwen3.5-4B, seed=42, LoRA r=16) online PyTorch GRPO evaluation across single-turn reasoning (Part II), multi-turn coding agents (Part III-A), 1,000-step co-evolution (Part III-B), and prefix-tree proof search (Part III-C).

Reward Paradigm (Identical `Qwen/Qwen3.5-4B` Base, `seed=42`)	Zero-Var Dead-Zone	Hack / Fallacy Rate	Part II: Math/IFEval (4-Bench Mean)	Part III-A: Agentic (4-Bench Mean)	Part III-B: Olympiad (4-Bench Mean)	Part III-C: Lean4/Tree (4-Bench Mean)
`Qwen3.5-4B-Base (Zero-Shot Control, 0-Step)`	—	19.2%	64.95%	51.83%	35.15%	31.50%
`RLVR-Binary-GRPO (DeepSeek-R1-Zero Style)`	38.4% – 46.8%	29.4% – 32.8%	73.78%	63.90%	43.45%	40.08%
`LLM-Judge-GRPO (Autoregressive Token Judge)`	12.6% – 15.7%	24.5% – 26.8%	72.33%	64.58%	46.08%	43.15%
`PRM-Linear-GRPO (Un-Gated Additive Process)`	5.6% – 8.2%	28.4% – 31.6%	76.12%	67.85%	44.33%	49.95%
System-One Static v1 (Ours: Static Score + Noul Gate)	2.4% – 4.8%	3.8% – 6.1%	83.20%	75.25%	54.00%	55.03%
System-One Dynamic v2 (Ours: Full Monograph SOTA)	1.8% – 4.0%	1.2% – 3.9%	85.90%	78.23%	59.28%	60.50%

05 Empirical Plates & Full Bilingual Manuscripts

Select any of the five monographs below to examine its primary 300-DPI experimental figures and access the compiled English and Chinese PDFs.

Part II Manuscript: Beyond Binary Verifiers and Verbose Judges: Single-Backbone System-One Process-and-Outcome GRPO for Reasoning and Instruction Alignment — English PDF (12 pp.) · Chinese Edition PDF (8 pp.) · DOI: 10.5281/zenodo.23441431

Part II Leaderboard — **Plate II.1.** Aggregate 4-benchmark LLM RL leaderboard and per-task accuracy across GSM8K, MATH-500, Countdown-24, and IFEval-Strict.

Part II Same-Backbone — **Plate II.2.** Strictly controlled Same-Backbone (`Qwen/Qwen3.5-4B`, `seed=42`) online PyTorch GRPO loss and strict process+outcome accuracy.

Part II Probing — **Plate II.3.** Live NVIDIA Blackwell 96GB System-One rollout probing across five reasoning and reward-hacking regimes.

Part II Dead-Zone — **Plate II.4.** Elimination of zero-variance GRPO group dead-zones ($-89.6\%$) and shortcut/yap-loop hacking ($-88.1\%$).

Part III-A Manuscript: Beyond Sparse Terminal Verifiers: Turn-Level Single-Backbone System-One Credit Assignment and Trajectory Gating for Multi-Turn Coding and Tool-Use Agents — English PDF (12 pp.) · Chinese Edition PDF (8 pp.) · DOI: 10.5281/zenodo.23442087

Part III-A Leaderboard — **Plate III-A.1.** Multi-turn agentic RL leaderboard across BFCL-v3, MBPP+, HumanEval+, and SWE-bench-Verified-Mini.

Part III-A Turn Probing — **Plate III-A.2.** Turn-level telescoping PBRS credit assignment and trajectory-minimum Noul suppression of `pytest` tampering.

Part III-A Same-Backbone — **Plate III-A.3.** Controlled Same-Backbone (`Qwen/Qwen3.5-4B`, `seed=42`) multi-turn online GRPO verification.

Part III-A KV Reuse — **Plate III-A.4.** Incremental multi-turn KV-cache prefix reuse ($84.6\%$) and component-wise ablations.

Part III-B Manuscript: Breaking the Static Verifier Ceiling: Zero-Overhead Self-Evolving System-One Judge and Policy Co-Evolution on a Single Shared Foundation Backbone — English PDF (12 pp.) · Chinese Edition PDF (8 pp.) · DOI: 10.5281/zenodo.23442137

Part III-B Leaderboard — **Plate III-B.1.** Hard competition mathematics leaderboard across MATH-500, AIME 2024/2025, AMC-12, and OlympiadBench.

Part III-B Long Horizon — **Plate III-B.2.** 1,000-step long-horizon training trajectory showing elimination of late-stage Goodhart collapse.

Part III-B ECE — **Plate III-B.3.** Contraction of Judge Expected Calibration Error ($\text{ECE}: 0.048 \to 0.011$) and pseudo-proof suppression.

Part III-B Same-Backbone — **Plate III-B.4.** Controlled Same-Backbone 1,000-step verification with $+0.02\text{ GB}$ pointer-head co-evolution.

Part III-C Manuscript: Prune Early, Verify Deep: Single-Backbone System-One Prefix Tree-GRPO and Speculative Noul Pruning for Long-Horizon Reasoning and Theorem Proving — English PDF (12 pp.) · Chinese Edition PDF (7 pp.) · DOI: 10.5281/zenodo.23442169

Part III-C Leaderboard — **Plate III-C.1.** Formal theorem proving & long-horizon reasoning leaderboard (MiniF2F-Lean4, ProofNet, MATH-L5, GPQA).

Part III-C Pruning — **Plate III-C.2.** Speculative Noul branch pruning ($-66.0\%$ wasted dead-end FLOPs) and test-time compute scaling.

Part III-C Probing — **Plate III-C.3.** Incremental $14.80\text{ ms/node}$ System-One prefix-tree probing on NVIDIA Blackwell 96GB.

Part III-C Same-Backbone — **Plate III-C.4.** Sibling-relative branch advantage ($-61.5\%$ gradient variance) vs. flat sequence GRPO.

Part I Manuscript: Beyond Brittle Code and Noisy Tokens: System-One Decision Models as Calibrated Dynamic-Curriculum Reward Judges for Continuous Control — English PDF (11 pp.) · Chinese Edition PDF (7 pp.) · DOI: 10.5281/zenodo.23439711

Part I Leaderboard — **Plate I.1.** Oracle-Normalized Score (ONS) across five canonical MuJoCo v5 continuous control benchmarks.

Part I Rubric Gate — **Plate I.2.** Zero-shot `Kev-4B` trajectory probing and multiplicative suppression of the Hopper Statue Hack.

Part I Curves — **Plate I.3.** Sample-efficiency and episode return convergence across all five MuJoCo v5 environments.

Part I Ablation — **Plate I.4.** Component ablations and single-prefill batch latency ($213.2\text{ ms}$ vs. $852.8\text{ ms}$).

06 Archival DOIs & BibTeX Records

All five preprints are permanently archived at CERN Zenodo under Creative Commons Attribution 4.0 International (CC-BY-4.0).

@article{zhang2026dcjevrl, title = {Beyond Brittle Code and Noisy Tokens: System-One Decision Models as Calibrated Dynamic-Curriculum Reward Judges for Continuous Control}, author = {Zhang, Ziyang}, year = {2026}, journal = {Zenodo Preprint}, doi = {10.5281/zenodo.23439711}, url = {https://doi.org/10.5281/zenodo.23439711} } @article{zhang2026dcjevgrpo, title = {Beyond Binary Verifiers and Verbose Judges: Single-Backbone System-One Process-and-Outcome GRPO for Reasoning and Instruction Alignment (DC-Jev-GRPO)}, author = {Zhang, Ziyang}, year = {2026}, journal = {Zenodo Preprint}, doi = {10.5281/zenodo.23441431}, url = {https://doi.org/10.5281/zenodo.23441431} } @article{zhang2026agenticjevgrpo, title = {Beyond Sparse Terminal Verifiers: Turn-Level Single-Backbone System-One Credit Assignment and Trajectory Gating for Multi-Turn Coding and Tool-Use Agents (Agentic-Jev-GRPO)}, author = {Zhang, Ziyang}, year = {2026}, journal = {Zenodo Preprint}, doi = {10.5281/zenodo.23442087}, url = {https://doi.org/10.5281/zenodo.23442087} } @article{zhang2026coevojevgrpo, title = {Breaking the Static Verifier Ceiling: Zero-Overhead Self-Evolving System-One Judge and Policy Co-Evolution on a Single Shared Foundation Backbone (CoEvo-Jev-GRPO)}, author = {Zhang, Ziyang}, year = {2026}, journal = {Zenodo Preprint}, doi = {10.5281/zenodo.23442137}, url = {https://doi.org/10.5281/zenodo.23442137} } @article{zhang2026treejevgrpo, title = {Prune Early, Verify Deep: Single-Backbone System-One Prefix Tree-GRPO and Speculative Noul Pruning for Long-Horizon Reasoning and Theorem Proving (Tree-Jev-GRPO)}, author = {Zhang, Ziyang}, year = {2026}, journal = {Zenodo Preprint}, doi = {10.5281/zenodo.23442169}, url = {https://doi.org/10.5281/zenodo.23442169} }