1 OpenLoopEvolve: A Verifiable Self-Evolution Framework for Loop Policies in Long-Horizon Complex Tasks Siqi Wang*, Xinlin Li*, Zhenglin Li, and Li Li arXiv:2608.09380v1 [cs.AI] 10 Aug 2026 Abstract--Long-horizon complex tasks require agents to re- ineffective calls consume resources needed later; moreover, a peatedly observe states, formulate plans, invoke tools, verify state change caused by one action may invalidate subsequent results, and recover from failures in continuously changing steps in the original plan. Observation granularity, recovery environments. However, such control experience often remains methods, and budget allocation therefore form interdependent confined to a single context or a fixed prompt, and is difficult to control relationships, and the final outcome depends on the accumulate and reuse across historical traces. This paper presents entire execution loop rather than only on single-step generation OpenLoopEvolve (OLE), a self-evolution framework centered quality. on the Loop Policy. OLE represents an agent's observation, planning, memory, action, verification, recovery, stopping, and Existing studies use external memory, linguistic reflection, budget control as portable policy assets with versions and and experience extraction to reuse past events or failures in lineages, and provides online and offline evolution modes that subsequent decisions [7]­[9]; executable skill libraries further can be selected according to practical needs: the online mode support the retrieval and composition of procedural behaviors triggers candidate generation based on feedback from continuous [10]. Meanwhile, declarative pipelines, code-based workflows, operation, whereas the offline mode searches for candidate poli- and modular agents bring structures external to the model into cies from archived traces and failure evidence. Both modes share automated optimization [11]­[14], and trajectory feedback has an evolution mechanism consisting of autonomous proposals by a also been used to continually revise multi-step reasoning pro- large language model, Champion­Challenger paired evaluation, cesses [15]. These directions improve experiential content and and robust release. Policies released online are activated at a execution structure, respectively, but they have not represented subsequent task boundary, monitored using subsequent feedback, the overall control relationships among observation, planning, and rolled back to their parent versions when degradation memory use, verification, recovery, stopping, and budget allo- conditions are met. On the simulated business benchmark YC- cation as a unified object. Memory can provide content from Bench, both modes improve aggregate task performance, task the past, and skills can reproduce local operations, but neither success rate, and risk metrics relative to a fixed initial Loop inherently specifies when to invoke experience, how to check Policy. The results indicate that treating the Loop Policy as execution results, or when to terminate. Although historical a governable asset can support the accumulation, comparison, traces can be stored or retrieved, their successes and failures release, and reuse of control experience and improve agent remain difficult to convert into comparable and transferable performance on long-horizon complex tasks. modifications to the complete loop. Index Terms--large language model agents, long-horizon tasks, The recently emerging perspective of Loop Engineering Loop Policy, self-evolution treats triggers, goals, verification, stopping rules, and memory organization as design elements of loops external to the model I. INTRODUCTION [16], providing a new level of analysis for the above problem. A loop object that can accumulate across runs also requires sta- Large language model agents are evolving from tools that ble control semantics, host interfaces, and version relationships generate one-shot answers into task executors capable of so that the same modification has comparable meaning across sustained action in external environments. Across a sequence different runs. Loop optimization must also separate candidate of decisions in a long-horizon complex task, an agent must generation from formal release: immediately rewriting the loop acquire observations, revise plans, invoke tools, verify results, without candidate validation may introduce incidental noise and, after a local failure, decide whether to retry, recover, or or tail degradation into subsequent tasks, whereas always stop. Interleaving reasoning and action allows agents to adjust retaining a fixed loop prevents the system from learning from their behavior based on environmental feedback [1], while in- recurring failures. Research on evidence gating has highlighted teractive evaluation has begun to expand from short processes the distinction between generated claims and verification states to cross-page operations and stateful business workflows [2]­ [17]; Champion­Challenger mechanisms also constrain ver- [5], as well as the simulated year-long business management sion replacement through candidate comparisons in predictive- studied by YC-Bench [6]. In these tasks, stopping prematurely model management and online AutoML [18], [19]. However, may leave unverified artifacts, whereas repeated planning and these mechanisms have not been applied directly to complete agent loops, nor do they provide a shared policy asset and *Siqi Wang and Xinlin Li contributed equally to this work and share first release semantics for offline historical trajectories and online authorship. operational feedback. Siqi Wang and Li Li are with the Department of Automation, Against this background, this paper focuses on two related Tsinghua University (e-mail: wang-sq24@mails.tsinghua.edu.cn; li- questions: first, how to separate the complete execution loop li@mail.tsinghua.edu.cn). Xinlin Li is with the Department of Industrial Engineering, Tsinghua University (e-mail: lix122@mails.tsinghua.edu.cn). Zhenglin Li is with Shenzhen International Graduate School, Tsinghua University (e-mail: li-zl23@tsinghua.org.cn). Corresponding author: Li Li. 2 OpenLoopEvolve Framework 1 experience construction 2 dual-mode Loop Policy evolution 3 evaluation & release online evolution long-horizon execution loop Champion­Challenger evaluation observe plan act verify matched context recent Traces LLM proposer candidate Bundle Champion Challenger State transition Online Challenger generation paired gain +1 = , , , = , , < = , , , = ,; , , - ; , , offline evolution , = ,max ; , , , Trace & evidence construction release decision & asset benefit evidence quality Trace archive evidence archived mutate/ robust gate resource tail risk records Traces recombine cost LLM proposer candidate Bundles Trace = , , , Offline population search robust release gate Evidence +1 = , , ,, = = = = , , , ,, = 1 =1 ,,, accumulate 4 environment activate instruct task interaction released Loop Policy Fig. 1. Overall OpenLoopEvolve framework. A released Loop Policy is activated at a subsequent task boundary, enters the host environment, and controls interactions in long-horizon tasks, producing attributable traces. The traces are converted into evolution evidence under a shared evidence contract and enter online or offline Loop Policy evolution. Candidates are evaluated against the Champion under identical conditions and, after passing the robust gate, are released as versioned policy assets and activated at a task boundary. from the host implementation and a single context to form read, compared, and reused independently of the original trace. an external policy asset that can be versioned, ported, com- A run's trace is then distilled into traceable evidence that pared, and reused; and second, how to autonomously improve points to policy modifications, after which a large language this asset using historical traces and current feedback while model autonomously proposes candidates. allowing only policies that have passed candidate validation, been released, and been activated at a task boundary to affect OLE provides online and offline evolution modes according future runs. Online updates need to avoid disrupting the state to the evidence source. The online mode generates a single of a task currently under execution, whereas offline search Challenger based on recent feedback; following paired evalua- needs to form comparable candidate evaluations from multiple tion and a robust gate, the Challenger is released as a successor heterogeneous traces. The two modes thus differ in when asset, activated at a subsequent task boundary, and then subject evidence becomes available but require consistent asset and to degradation monitoring and rollback. The offline mode release semantics. The first question determines the object that constructs a candidate population from archived traces and serves as the unit of loop evolution, and the second determines performs multi-generation search through batch evaluation, how evidence drives policy changes and when those changes elite retention, and autonomous mutation and recombina- can be released and activated. The objective of this paper is not tion. The two modes can be adopted separately according to allow an agent to rewrite itself arbitrarily within a single run, to run continuity, budget, and candidate-validation timing, but to place policy search outside the execution path so that while sharing the boundary of "evidence­candidate­candidate candidates, evaluation evidence, and formal versions remain validation­release­activation": candidate generation belongs clearly separated. to the search process, only policies satisfying the candidate- validation requirements can be released, and only a released To address these questions, this paper presents OpenLoopE- policy activated at a task boundary can enter the execution volve (OLE), a verifiable self-evolution framework for Loop path. This forms a closed loop from historical interactions to Policies in long-horizon complex tasks. OLE organizes ob- subsequent policies, as illustrated in Fig. 1, allowing control servation, planning, memory, action, verification, recovery, experience to evolve continually while preventing provisional stopping, and budget control into an externalized Loop Policy, candidates from directly altering the execution path. and encapsulates the policy, its applicability conditions, and its version provenance in a Bundle within a policy lineage. The main contributions of this paper are summarized as The Loop Policy constrains a run through general control follows: semantics without binding it to the specific model and tool implementation of the host, enabling loop experience to be 1) We propose the Loop Policy and Bundle protocol, which represents complete loop control as an external policy asset with explicit interfaces, versions, and lineages, 3 enabling historical control experience to be preserved, objective, however, remains to improve or evaluate agent be- compared, exchanged, and reused across runs. havior within a given execution structure; how observations are 2) We propose online and offline self-evolution methods organized, when planning or recovery occurs, how outcomes based on trace evidence. A large language model au- are verified, and when execution stops are typically determined tonomously proposes candidates, while paired evalua- by the particular method or host framework. This leaves room tion, robust release, task-boundary activation, and roll- to investigate how these control rules can be separated from back govern their entry into the execution path. To our the implementation details of an individual run and treated as a knowledge, this work is the first to apply the Champion­ unified object for comparison and optimization. Enabling loop Challenger release mechanism to both offline and online control rules to improve continually across runs also requires self-evolution of externalized Loop Policies. addressing how historical interactions can be accumulated and 3) We evaluate OLE on YC-Bench. Results obtained with brought back into subsequent decisions. fixed models, benchmark configurations, and official seeds show that both online and offline modes outper- B. Memory, Reflection, and Reuse of Trajectory Experience form the fixed initial Loop Policy in long-term returns, task success rate, annual survival, and risk metrics. Memory and reflection provide the main means for agents to accumulate experience across rounds. Their shared premise is The remainder of this paper is organized as follows. Section to preserve historical interactions beyond the limited context II reviews research on long-horizon interactive agents, reuse window in an external medium and retrieve, summarize, or of memory and trajectory experience, agent workflow opti- reorganize them during subsequent decisions, thereby main- mization, and self-evolution. Section III defines long-horizon taining consistency in long-term behavior. Park et al. [7] con- complex tasks, the loop interaction process, the trace, evolution nect the recording of experiences with reflection and planning, evidence, Loop Policy, and Bundle. Section IV introduces enabling agents to derive higher-level insights from past events the shared evolution framework of OpenLoopEvolve and its and use them to guide subsequent behavior; this demonstrates online and offline implementations. Section V presents the that historical trajectories can contribute to continual decision- experimental results and analysis. Section VI concludes the making beyond serving as raw conversation records. paper. The code for this paper can be found at https://github. com/yoyoshikc/OpenLoopEvolve. Existing methods differ in how they preserve and use experience. Shinn et al. [8] convert task feedback into lin- II. RELATED WORK guistic reflections and store them in episodic memory, allow- ing subsequent trials to draw on failures without updating A. Long-Horizon Interactive Agents and Loop Execution model parameters; Zhao et al. [9] extract natural-language experiences from a set of training tasks and jointly retrieve Multi-round interaction with an environment is a fundamen- abstract insights and similar successful trajectories at inference tal capability of large language model agents in complex tasks. time. In contrast to textual experience, Wang et al. [10] store Unlike one-shot answer generation, an agent must receive successful programs generated from environment feedback in environment states, select actions, and use new feedback a retrievable skill library, allowing procedural behaviors to to advance its objective over a sequence of decisions; task be composed and reused in new Minecraft worlds. These performance therefore depends on consistent reasoning and ex- methods show, respectively, that historical trajectories can ecution across steps. Existing comprehensive benchmarks have be distilled into reflective text, experiential knowledge, or examined these capabilities in diverse interactive environments executable skills. and have further recorded agents' intermediate progress before task completion [2], [3]. These mechanisms substantially expand agents' ability to reuse past experience, but their update targets are primarily One line of research on this process constructs an execution memory content, linguistic experience, or task skills, and loop in which reasoning and action are interleaved. Yao et they generally do not directly represent the overall control al. [1] organize language reasoning, external actions, and relationships among observation, planning, verification, recov- environment observations within the same trajectory, enabling ery, stopping, and budget allocation. This paper does not the model to adjust subsequent judgments based on execution seek to add another form of memory content; instead, it outcomes. Another line of research embeds such loops in distills evidence from traces that supports policy comparison workflows involving real state changes: Zhou et al. [4] evaluate and uses this evidence to modify how the complete loop is cross-page operations and functional correctness in web envi- controlled. This distinction leads further to the question of ronments, while Yao et al. [5] further evaluate task comple- agent workflows and their self-evolution. tion in interactions involving tools, users, and domain states. Correspondingly, long-horizon benchmarks have also begun to C. Agent Workflow Optimization and Self-Evolution emphasize the cumulative consequences of control decisions: He et al. [6] test long-term planning and consistent execution As agent systems have expanded from single prompts through simulated business operations sustained over a year, into composite programs containing model calls, tools, and rather than evaluating only a single-round response. control flow, the object of study has gradually shifted from prompt optimization to the optimization of workflows and These studies establish the closed-loop interaction paradigm agent structures. Khattab et al. [11] represent language model for long-horizon tasks and demonstrate the roles of planning, pipelines as composable declarative modules and compile their tool use, and state feedback in task completion. Their primary 4 parameters against specified metrics; Hu et al. [12] instead externalized Loop Policy, enabling loop-control experience represent agent systems as code, enabling a meta-agent to from long-horizon tasks to be verified, released, and reused generate and search for new agent programs, with a code as an independent asset. space that can in principle cover prompts, patterns of tool use, and workflow compositions. These studies show that execution III. PROBLEM FORMULATION structures external to the model can themselves become objects of automatic optimization. A. Long-Horizon Tasks and the Loop-Control Problem In terms of the evidence used for optimization, existing work Sustained execution of multi-round tasks in interactive can be summarized along two paths: offline structural search environments is an important test of an agent's reasoning, and continual improvement driven by runtime feedback. Of- decision-making, and execution capabilities, and has become fline search uses existing tasks, execution evaluations, and ac- a major focus of general-agent research and evaluation [2], cumulated search experience to produce candidate structures: [3]. Such tasks require an agent to engage in closed-loop Zhang et al. [13] model code-represented Agent Workflows interaction with an external environment over successive de- as a search space and iteratively modify workflows using cision rounds toward a given objective: the agent selects an execution feedback and tree-structured experience; Shang et action based on its current observation, the action changes the al. [14] abstract planning, reasoning, tool use, and memory environment state and produces new feedback, and subsequent into modules with unified interfaces and search for agent decisions therefore depend on prior interactions. Existing designs through evolution and recombination. By comparison, research characterizes the long-horizon nature of these tasks trajectory-driven self-evolution research places greater empha- primarily at two levels: solution mechanisms and evaluation sis on using newly generated interaction feedback to adjust protocols. At the solution-mechanism level, Yao et al. [1] subsequent solution processes; Guo et al. [15] extract feedback organize task execution as closed-loop interaction, enabling from multi-step reasoning trajectories and iteratively optimize an agent to update its plan in response to feedback and the reasoning process. The former focuses on searching for handle exceptions. At the evaluation-protocol level, WebArena structures in controlled evaluation environments, whereas the and Tau-bench [4], [5] construct tasks as realistic interactive latter focuses on allowing runtime experience to continually workflows with multistep dependencies and determine task influence subsequent behavior; its focus nevertheless remains completion from functional correctness or the post-execution iterative optimization of the reasoning process and does not environment state. The former reflects closed-loop decision- yet address policy release governance for deployment. To- making and state dependence during execution, whereas the gether, these two paths move agents from fixed designs toward latter ensures that task outcomes can be verified through the experience-driven continual optimization. external environment. The recent emergence of Loop Engineering further con- Following the closed-loop interaction perspective, we view centrates attention on loop structures external to the model. the solution of a long-horizon complex task as an execution Macedo [16] organizes trigger conditions, objectives, verifica- loop in which an agent repeatedly cycles through reasoning, tion, stopping rules, and memory into reusable loop specifi- action, and observation of the environment, and we focus cations, while Huang et al. [17] emphasize that only evidence on the mechanisms that control observation, planning, action, satisfying a verification gate can drive state transitions in the verification, recovery, and stopping within this loop. We agent lifecycle. These studies extend workflow optimization to define a long-horizon complex task as one whose solution the systematic design of complete execution loops and begin process cannot be reliably reduced to a single model gen- to address verification and state governance within them. In re- eration without environment interaction, but instead requires lated work, the Champion­Challenger paradigm has been used an agent to execute multiple consecutive decision steps with for the continual comparison and replacement of production state dependencies under finite resource constraints, adjusting predictive models [18] and has been further developed into subsequent behavior according to tool returns and environment a hyperparameter-selection algorithm that concurrently main- feedback until it produces a verifiable task outcome. Formally, tains one Champion and multiple online Challengers [19]. Ex- we describe a long-horizon complex task using the following isting methods, however, primarily compare predictive models executable contract: or learning configurations. Loop Engineering itself has not yet established unified object boundaries and interface standards; g = d, Ug, Mg, Sg, Vg, g , g G. (1) most existing optimization results remain specific programs, task structures, or configurations internal to a framework, Here, G denotes the space of long-horizon complex tasks. d de- while offline search and online runtime feedback still lack a scribes the task objective, expected artifact, and initial condi- shared policy asset and release semantics. To address this gap, tions, providing an anchor for planning and replanning within this paper represents loop control logic as a versioned and the loop. Ug specifies the set of executors permitted by the portable Loop Policy, uses a unified protocol to accommodate task, thereby restricting the tool calls and environment actions both offline and online candidates, and governs their evo- available to the loop. Mg provides the evaluation metrics that lution through Champion­Challenger comparison, evidence- determine how execution progress, outcome quality, and re- constrained release, and rollback. To the best of our knowledge, source cost are measured. Sg restricts the evidence sources that this paper is the first to apply a Champion­Challenger release may support a judgment, such as environment observations, mechanism to the offline and online self-evolution of an run logs, and task artifacts, preventing task completion from being established solely by the model's own statements. Vg 5 specifies when and how verification is performed and feeds its constrained optimization problem, cVg (; g, z) denotes the results back into replanning, recovery, and stopping. g is the verification-risk vector determined by verification protocol resource-budget vector comprising upper bounds on quantities Vg, g denotes the corresponding admissible upper bounds, such as interaction rounds, tool calls, time, and tokens, and b(; g, z) denotes the resource-consumption vector, and thus bounds retries and exploration. These six fields specify the denotes elementwise vector comparison. This optimization loop's objective, capability boundary, measurement criteria, problem jointly characterizes outcome quality, verification grounds for judgment, verification mechanism, and resource reliability, and resource boundaries. The observations, actions, boundary, respectively. and environment feedback produced during Loop Policy ex- ecution further form interaction traces, providing empirical Based on this formal definition, we further characterize an grounds for analyzing existing Loop Policy behavior and sup- agent's loop interaction process within a single run to make porting subsequent evolution. The next subsection accordingly explicit the dynamic relationships among observations, internal defines a trace and its evidence form. states, actions, and environment feedback. Before giving its structured components, we use to denote the Loop Policy B. Traces and the Evidence Contract space abstractly and let denote the Loop Policy that constrains the current execution loop; its eight-component The continual evolution of a Loop Policy depends on the structure is defined in Section III-C. The loop-control process effective use of historical runs, requiring the execution process can be abstracted as to be formalized as a trace that links the task, Loop Policy, and outcome. Let the trace of the ith run be xt+1 = f(xt, ot, at, et), at+1 h(xt+1), (2) where t denotes the current interaction step; xt X de- i = (gi, v(i), Hi, yi), Hi = {(ot, at, et)}Tt=i 1, (4) notes the internal control state at step t; ot O denotes the observation that the agent receives from a tool or the where gi denotes the task processed in the ith run, v(i) denotes environment at step t; at A denotes the action executed at the stable version index of the registered Bundle used in step t; Eenv denotes the environment-event space, and et Eenv that run, and v(i) denotes the Loop Policy in that Bundle. denotes the environment event caused by action at; denotes The stable index v(i) can be used to retrieve the complete the Loop Policy that constrains loop behavior; f denotes asset Bv(i), including its applicability conditions and version the update function that, under Loop Policy , computes the provenance. Hi denotes the interaction history composed of successor state from xt, ot, at, and et; xt+1 denotes the observations, actions, and environment events, Ti denotes the next control state produced by f; h denotes the decision trace length, and yi denotes the run outcome. This represen- function that selects an action from xt+1; and at+1 denotes tation explicitly preserves the mapping between policy-asset the next action produced by h. An action may be a tool identity and the corresponding interaction outcome, ensuring call, information acquisition, artifact modification, outcome that historical performance can be accurately attributed to a verification, or stopping decision, and its available range is specific registered Bundle version. jointly constrained by Ug in the task contract and the host's capabilities. The state xt is an abstract representation of the After obtaining attributable traces, we must further distill control information required to run the loop and need not be a from them evolution evidence that can guide adjustments to fully observable Markov state. Factors including observation, the Loop Policy. We define such evidence as a testable basis planning, memory use, reflection, risk handling, evaluation, for modifying a specific Loop Policy component, inferred from and budget allocation affect subsequent state updates and a set of traces that are attributable, whose outcomes are verifi- action selection through f and h. able, and whose evidence sources conform to the task contract. Each evidence item must specify the recommended direction Having characterized the loop interaction process, we must of adjustment, the associated performance change, the policy also specify the basis for evaluating a Loop Policy and its component concerned, and the original traces supporting the optimization objective. Let q(; g, z) R denote the value judgment. Formally, the evidence-extraction operator maps score attained by Loop Policy on task g under controlled a trace set D to run condition z Z, where Z denotes the run-condition space and q is determined by the evaluation metrics Mg E = (D) = {j = (j, j, uj, Ij)}Jj=1, (5) and verification protocol Vg in the task contract. Under these conventions, Loop Policy optimization seeks, for the same task where j denotes the jth item of evolution evidence and and controlled run condition, a Loop Policy with a higher value J denotes the number of evidence items; j {+, -} score that satisfies the verification requirements and resource denotes the direction of the evidence: positive evidence in- boundary. This objective can be formalized as dicates that the relevant behavior pattern is consistent with improved performance and may support Loop Policy reuse, arg max q(; g, z) whereas negative evidence indicates that the relevant behavior pattern is consistent with failure or performance degradation and may support Loop Policy revision; j denotes a metric change relative to a control run or a statistical reference for s.t. cVg (; g, z) g, (3) comparable traces; uj denotes the Loop Policy component targeted by the evidence, taking a value corresponding to b(; g, z) g, observation, planning, memory, action, verification, recovery, where is the Loop Policy space formally defined in Section III-C, denotes an optimal Loop Policy for this 6 stopping, or budget control as defined in the next subsection; loop-policy-v2.1 and Ij {i | i D} denotes the set of global trace indices supporting the judgment. observation: select task-relevant feedback planning: update plan after each observation In summary, the trace and evidence contract transforms memory: retrieve reusable trace evidence historical interactions into attributable and verifiable evolution action: choose tool or agent action evidence that can be used to adjust the Loop Policy, laying verification: require external evidence the foundation for defining the modifiable and reusable Loop recovery: retry | reroute | replan Policy object in the next subsection. stopping: verified | budget exhausted budget control: allocate calls, time, and tokens C. Loop Policy Fig. 2. A YAML-style example of a Loop Policy. The boldface fields Run evidence must act on an explicit loop-control object be- correspond to the eight categories of loop-control rules: observation, planning, fore it can be converted into reusable Loop Policy experience. memory, action, verification, recovery, stopping, and budget control. Loop Engineering extends the scope of engineering to the external agent loop and systematically designs its objectives, corresponding to the vth registered Bundle version as a Bun- feedback, verification, stopping, and cross-run experience [16]. dle: Building on the preceding abstract type declaration, we define the set of external rules that controls how an agent advances through a complete execution loop as a Loop Policy, whose eight-component structure is denoted by = obs, plan, mem, act, (6) Bv = v, v, v , (7) ver, rec, stop, bud , where denotes the Loop Policy space; obs governs the where v is the stable version index of a registered Bundle, acquisition and filtering of observations, plan governs plan v denotes the Loop Policy of this version, v Z denotes generation and updating, mem governs memory writing, re- the set of applicability conditions for this Loop Policy, and trieval, and use, act governs action and tool selection, ver v denotes the version provenance, including the stable index governs outcome verification, rec governs recovery from pv of the direct parent version and the evolution evidence on failure, stop governs the loop's stopping conditions, and bud which the version is based; the initial Bundle has pv = governs resource-budget allocation and adjustment. Together, and no supporting evolution evidence. Together, these three these eight components determine the state-update relation components describe which Loop Policy is used, under what f and action-decision relation h in Eq. (2), allowing a conditions it is executed, and its version provenance, allowing Loop Policy to cover the complete decision process from the Loop Policy to be stored and reused as a complete object. incorporating environment feedback into the control state to Registration assigns a stable index when a valid candidate producing the next action. Bundle is formed so that it can enter evaluation history and lineage; registration itself does not mean that the candidate has Fig. 2 presents a simplified Loop Policy example in a passed the gate, been released, or been activated. Hereafter, k YAML-style format. This representation organizes the eight is reserved for evolution rounds or offline generation indices categories of loop rules into a readable and modifiable ex- and no longer serves as a stable version identity. ternal structure, allowing Loop Policy content to be stored, compared, and reused independently of any single-run context. Bundles are compared under shared applicability conditions z par cand, where the subscripts par and cand denote To support the reuse of loop-control experience across tasks, the parent and candidate versions, respectively, and par and we design a Loop Policy as an external asset independent of cand are the corresponding Loop Policies. For each noninitial a specific host agent and use a unified representation to carry version with pv = , the version provenance v links it to its control semantics. A Loop Policy can be integrated as a its direct parent version and to the run evidence supporting pluggable object into different agent frameworks through an the change. The directed relationships from parent versions to adaptation interface, without binding its content to a particular successor versions form a policy lineage, allowing historical host implementation, thereby supporting Loop Policy replace- experience to accumulate along version relationships and sup- ment, comparison, and evolution across agent frameworks. porting subsequent candidate comparison, version selection, This externalized design provides the basis for the versioned and outcome tracing. asset representation in the next subsection. This section has defined, in sequence, the loop interaction D. Asset Representation of a Loop Policy process, traces and evolution evidence, the Loop Policy, and its asset representation, beginning with the long-horizon complex As a control object that can be independently compared task. These definitions specify the objects of self-evolution, the and reused, a Loop Policy also requires a stable asset rep- sources of experience, and the basis for comparison, providing resentation. Reusing a Loop Policy across runs requires the a unified notation for the next section's presentation of online Loop Policy itself, its applicability conditions, and its version and offline evolution, the candidate lifecycle, and the robust provenance to be jointly identifiable. We represent the asset release gate. 7 IV. OPENLOOPEVOLVE METHOD where i,j is the candidate's absolute value-score change relative to the Champion, ri,j is the relative change, and > 0 Section III formally defined long-horizon complex tasks, ensures numerical stability when the Champion value score traces, evolution evidence, and Loop Policies. Building on is close to zero. In each paired run, the Loop Policy in the these definitions, OpenLoopEvolve converts traces accumu- corresponding Bundle controls the execution loop in Eq. (2). lated across runs into a basis for modifying Loop Policies and The value score q remains determined by Mgi and Vgi in optimizes an external Loop Policy rather than the parameters the task contract, while the action scope and resource upper of the base model. Starting from the current Champion, the bounds comply with Ugi and gi , respectively. Holding the method constructs evidence, generates candidates, performs task, model, tools, and resource conditions constant controls paired evaluation, and applies constrained release to produce for the primary external sources of variation, allowing paired a successor version. Depending on the evidence source and changes to reflect the effect of Loop Policy modifications update timing, this common framework can be instantiated as more directly. The valid relative changes for candidate j form either online or offline evolution. the sample rj = {ri,j} and, together with the verification risk cVgi in Eq. (3), resource consumption b, failure states, A. Overall OpenLoopEvolve Framework and evidence validity, constitute the paired evaluation Pk,j. For runs with termination failures or invalid evidence, the The two evolution modes of OpenLoopEvolve share the evaluation protocol uses a bounded failure-aware change and same processing chain from run experience to policy assets. retains the corresponding validity and reliability states. Thus, Let Dk denote the set of traces used in evolution round k. The PairEval(Bc, Bk,j) in the algorithms below always denotes corresponding evidence-construction process is paired evaluation of complete Bundles under shared applica- bility conditions, rather than a comparison of two bare policies Ek = (Dk), Dk {Doff, Dkon}, (8) without regard to applicability conditions. where follows Eq. (5); Doff and Dkon are attributable traces Paired evaluation does not directly replace the current policy available from archives and online operation, respectively. The according to a single mean score; it provides evidence for online mode retains the round index and uses Ek = (Dkon); constrained release. Let b, e, t, and c indicate whether because the offline archive remains fixed across generations, the four constraint categories of benefit, evidence quality, tail (Doff) is abbreviated as E below. A trace can enter Dk only if risk, and resource cost are satisfied, respectively. The release its outcome has been verified by Vgi and its source belongs to gate can be expressed as Sgi . Candidate generation reads j = (j, j, uj, Ij), where uj identifies a component in Eq. (6); GoLnLM returns one registered Gate(Pk,j , ) = accept Bundle, whereas GLofLfM and VLoLffM return sets of registered Bundles; a proposal that fails to form a valid Bundle yields (10) the empty value or is excluded from the set. A candidate (Pk,j , ) = 1, may be released as a successor asset only after passing paired evaluation and the gate. The host activates a released policy Jgate only at a task boundary, and only the activated policy controls environment interaction according to Eq. (2) and produces new where Jgate = {b, e, t, c} corresponds to benefit, evidence traces as defined in Eq. (4), thereby forming the closed loop quality, tail risk, and resource cost; (Pk,j, ) {0, 1} shown in Fig. 1. indicates whether the constraint of category is satisfied; and = {} is the gate configuration. e checks the evidence For readability, Table I summarizes the key symbols used permitted by Sgi , t enforces cVgi gi , and c enforces throughout the problem formulation and method in the order b gi . The number of pairs, win rate, confidence interval, of task and interaction, trace and evidence, policy assets, and task failures, and tail losses may be used as internal evaluation evolution and release. Indices, temporary variables, and hy- quantities. A Challenger is eligible for release only if all perparameters used only within a single equation or algorithm enabled constraints are satisfied, and the configured thresholds are defined locally. cannot relax the verification and resource boundaries of the task contract. Champion­Challenger paired evaluation provides a common comparison basis for both modes [18], [19]. Let the current Within this common framework, the online and offline Champion Bundle be Bc = c, c, c , and let candidate Bun- modes are suited to different forms of available run expe- dle j generated in round k be Bk,j = k,j , k,j , k,j Bvk,j , rience. When the task distribution continues to change or where (k, j) is the candidate's search coordinate and vk,j is the new feedback must be incorporated promptly, the online mode stable version index assigned when the candidate is formed. can perform staged updates. When archived traces cover the The two are executed separately only on the same task gi and main run states of the target task, the offline mode can under the same controlled run condition zi,j c k,j. For conduct a concentrated search over multiple policy directions. pair i of candidate j, the value-score difference and relative The application selects the mode according to the scope of change are defined as available traces, feedback timeliness, and update requirements; the system does not switch modes automatically. The modes i,j = q(k,j ; gi, zi,j ) - q(c; gi, zi,j ), differ in candidate-search procedures and update timing, while their policy comparison and release eligibility are governed ri,j = i,j , (9) uniformly by Eqs. (9) and (10). max{|q(c; gi , zi,j )|, } The evaluation histories of the two modes are explicitly typed. Let R = Ron Roff be a disjoint union. Each record 8 TABLE I MAIN NOTATION AND MEANINGS Category Symbol Meaning Task and g = d, Ug, Mg, Sg, Vg, g interaction Executable contract for a long-horizon task, comprising its objective, xt, ot, at, et; f, h permitted executors, evaluation metrics, evidence sources, verification Traces and z Z; q(; g, z); cVg , g; b, g protocol, and resource budget evidence i = (gi, v(i), Hi, yi) Control state, observation, action, and environment event at step t; state- D, E = (D) update and action-decision functions constrained by the policy Policy assets j = (j , j , uj , Ij ) Controlled run condition; value score; verification risk and its upper bound; resource consumption and its upper bound Evolution and release Bv = v, v, v Attributable trace of run i, including the task, stable Bundle version, v, v interaction history, and run outcome Doff, Dkon; E , Ek Set of attributable traces with verified outcomes and compliant evidence GLonLM, GLofLfM, VLoLffM sources; the evolution-evidence set; and the evidence-extraction operator R = Ron Roff Direction, metric change, target policy component, and supporting trace indices of evolution-evidence item j Bc, Bk,j Bvk,j i,j , ri,j , Pk,j Loop Policy and its policy space; the policy comprises rules for obser- , , Gate vation, planning, memory, action, verification, recovery, stopping, and Ck, Fk, Pk budget control Bundle corresponding to stable version index v, comprising a Loop Policy, a set of applicability conditions, and version provenance Bundle applicability conditions and version provenance containing the parent version and supporting evidence Archived offline traces, traces visible in online round k, and their corresponding evolution evidence Online single-candidate generation, offline initial-candidate-set generation, and offline candidate mutation and recombination operators Mode-typed evaluation histories; the online component records candi- dates, evaluations, and release decisions, whereas the offline component records candidate sets and evaluation mappings Champion and Challenger Bundle j in round k; vk,j is its stable version index (online: j = 1, Bk Bk,1) Absolute and relative value-score changes for pair i, and the complete paired evaluation of a candidate Release constraint of category , its configuration, and the aggregate release-gate operator Candidate set, retained set comprising the fixed Champion and selected nondominated candidates, and cumulative paired-evaluation mapping at offline generation k in the online component Ron is (B, P, ), which stores a where Bk = k, k, k is the current Champion held at candidate Bundle, its paired evaluation, and the release deci- online update time k, and R