The Sleeping Agent: What Gist-Based Context Compression Loses and Why N. E. Kyrkewood (Independent Researcher) https://github.com/kyrkewood/sleeping-agent arXiv:2608.11775v1 [cs.AI] 12 Aug 2026 Abstract. Gist-based context compression--summarising older conversation history into compact representations--is a common approach in long-horizon language model agents, yet its effect on different types of memory retrieval is poorly understood. We use Salience-Weighted Consolidation (SWC), a biologically-inspired compression framework motivated by sleep-based memory consolidation, as a diagnostic probe to study when gist compression helps and when it hurts. SWC scores conversation history by salience, partitions it into priority tiers, and applies structured gist abstraction to mid-priority content. Evaluating four conditions on all ten LoCoMo conversations--1 935 matched text-only questions in total, 1 501 used in the primary aggregate after excluding Category 5 (adversarial) questions--at temperature 0, we find a consistent task-type interaction: gist compression substantially outperforms truncation on multi-hop reasoning and single-hop factual questions, but temporal questions remain substantially harder under compression, with compressed conditions scoring well below the full-context reference on the conversations where both are evaluated. We trace this failure to a specific mechanism: the gist abstraction prompt preserves relational and event structure while discarding dates and times. A preservation analysis across all ten conversations confirms the mechanism: an approximately 20-fold increase in temporal expression preservation (3.05%62.39%) with a one-sentence prompt modification, while named entity and event preservation rates barely change (×1.02 and ×1.11), demonstrating that the fix is a precision instrument. The prompt modification recovers +0.314 [0.254, 0.375] judge accuracy on category-2 (temporal) questions in the matched set. Code and results: https://github.com/kyrkewood/sleeping-agent. 1 Introduction We use Salience-Weighted Consolidation (SWC)--a biologically-inspired compression framework--as a di- Memory management in long-horizon language model agnostic probe. SWC scores context chunks by esti- agents involves a recurring design decision: when conver- mated salience, partitions them into priority tiers, and sation history grows too long to fit in the active context applies structured gist abstraction to mid-priority con- window, what should be kept, compressed, or discarded? tent. We identify a consistent task-type interaction: gist A common practical approach is reactive summarisation: compression outperforms truncation on multi-hop and when context exceeds a threshold, older history is con- single-hop questions but fails specifically on temporal densed by a summarisation model and substituted back. ones. A preservation analysis confirms the mechanism: This applies compression uniformly, regardless of what a temporal-protecting prompt modification causes an ap- the history contains or what questions the agent will later proximately 20-fold increase in temporal preservation need to answer. while barely affecting named entity and event preserva- tion (×1.02 and ×1.11). That single prompt change re- Recent work has shown that structured memory systems covers +0.314 judge accuracy on category-2 (temporal) substantially outperform simple compression. LightMem questions in the matched set, with the direction positive (Fang et al., 2025), Mem0 (Chhikara et al., 2025), Mem- in all ten conversations. The failure mode is a correctable Machine (Wang et al., 2026), and related systems report design omission. substantially higher LoCoMo scores than truncation and sliding window, often in the 0.7­0.9 range, though re- The finding that temporal anchors are systematically and sults come from systems of varying architectural com- selectively lost under generic gist compression suggests a plexity and evidence quality ranging from peer-reviewed design principle for gist-compression components across (LightMem, ICLR 2026) to self-reported preprints. These memory architectures: temporal anchors should be ex- systems represent a clear advance, but their complexity plicitly protected rather than left to generic abstraction makes it difficult to understand which components of prompts. memory management matter and why. 1 2 Background isolation. SleepGate (Xie, 2026) uses learned KV cache gating to 2.1 Two-Stage Model and Selective Replay target proactive interference architecturally. Adaptive Focus Memory (Cruz, 2025) uses multi-fidelity context The two-stage model (McClelland et al., 1995) proposes packing based on semantic relevance and temporal decay. that memories are initially encoded in the hippocampus ACON (Kang et al., 2025) optimises context compression and gradually transferred to the neocortex during sleep through natural language feedback. ENGRAM (Patel and via selective replay. Memories associated with reward, Patel, 2025) proposes lightweight memory orchestration novelty, or high salience are preferentially reactivated. using structured extraction and retrieval. 2.2 Forgetting as Function 4 Salience-Weighted Consolidation The synaptic homeostasis hypothesis (Tononi and Cirelli, 4.1 Design Principles 2006) proposes that sleep reverses waking-induced synap- tic potentiation through global downscaling, preserving SWC draws three principles from sleep neuroscience: stronger synapses while pruning weaker ones. Forgetting proactive scheduling (consolidation at idle windows, not is an active noise-reduction process. capacity limits), salience-weighted retention (chunks scored and partitioned), and staged compression 2.3 Episodic Specificity and Gist (salience-scoring pass followed by gist abstraction). Sleep converts episodic memories (specific events, times, 4.2 Stage 1: Salience Scoring contexts) to more semantic representations. This conver- sion is adaptive for long-term knowledge but lossy for Each session chunk receives a score from three signals: specific detail. Named entities and event mentions sur- Downstream similarity (0.4): Cosine similarity vive at higher verbatim rates than temporal expressions between chunk and subsequent turn embeddings, in generic gist summaries (8% and 5% versus 3%). computed locally using all-MiniLM-L6-v2 via The operative distinction is temporal versus non-temporal, sentence-transformers (no embedding API called; not episodic versus semantic. only Anthropic APIs used for generation and judging). Sessions with temporally specific facts may score low de- 3 Related Work spite later relevance--consistent with the temporal failure mode. 3.1 Context Management Recency (0.3): Session index ÷ total sessions. Information density (0.3): Length-normalised named The `lost in the middle' phenomenon documents perfor- entity and number count. mance degradation as context length increases (Liu et al., Scores partition chunks: high priority (0.6, verbatim), 2024). Proactive interference is an additional source of mid priority (0.3­0.6, gist-compressed), low priority degradation (Wang and Sun, 2025). Truncation is recency- (<0.3, discarded). The two most recent sessions and biased; sliding window summarisation applies uniform session 0 are always high priority. If history exceeds compression regardless of content importance. the 4 000-token budget, high-priority chunks are dropped lowest-salience-first. 3.2 Memory Systems 4.3 Stage 2: Gist Abstraction -- SWC-Full LightMem (Fang et al., 2025) reports substantial ac- curacy and token-efficiency gains on LoCoMo using a Mid-priority chunks are compressed using Claude Haiku staged architecture with sleep-time offline update. Mem0 (claude-haiku-4-5-20251001, temperature 0): (Chhikara et al., 2025) uses dynamic extraction, consolida- tion, and retrieval with a graph-memory variant, reporting Preserve: factual information about the speakers, a 26% relative LLM-as-judge improvement over OpenAI causal relationships, decisions and their rationale, on LoCoMo. MemMachine (Wang et al., 2026) reaches open commitments or plans. Discard: small talk, 0.917 on LoCoMo using ground-truth-preserving episodic repeated information, emotional reactions without storage with contextualised retrieval. factual content. Output a concise paragraph of 3­5 SWC is not proposed as a competitor to these systems but as a controlled probe to study gist compression effects in 2 sentences maximum. The matched set--the intersection of questions answered by all four compressed conditions--comprises 1 935 text- No instruction retains dates or times; the preservation only questions across all ten conversations. The primary analysis confirms that only 3.05% of temporal expressions categories 1­4 aggregate uses 1 501 of these. Bootstrap survive verbatim. CIs for Table 1 use the 1 501-question subset; per-category CIs use each category's own matched question count. 4.4 Stage 2: Gist Abstraction -- SWC-Temporal 5.2 Conditions Identical to SWC-Full except for one added sentence at the prompt start: Truncation: Most recent tokens within a 4 000-token budget, pinning session 0. Sliding window: Condenses You MUST preserve verbatim: all specific dates, history exceeding the budget using Haiku instructed to times, durations, ages, and temporal expressions retain key facts, events, and decisions. SWC-Full: Two- (e.g. `7 May', `last Tuesday', `at 9pm'). stage salience-scoring and gist abstraction pipeline (§4.3). SWC-Temporal: Same pipeline with temporal-anchor SWC-Temporal uses a separate REM cache; salience scor- protection (§4.4). Full context: Entire uncompressed ing is identical to SWC-Full. dialogue (no budget constraint); performance ceiling only, evaluated on conversations 0­1. 4.5 Scheduling 5.3 Category Mapping comes from SWC-Full uses salience-pressure and idle-window trig- gering with consolidation every three sessions. SWC- The authoritative mapping Reactive (token-threshold only) produced near-identical task_eval/evaluation.py: scores in early runs, consistent with offline benchmarks providing no genuine idle windows. Scheduling is de- ID Type Notes signed for online deployment and is not evaluated here (Appendix C). 1 Multi-hop Partial F1 Implementation: Salience scoring runs on commodity 2 Temporal `When' questions CPU hardware,1 taking 30 min­2.5 h per conversation. 3 Open-domain Commonsense infer- 5 Experimental Setup ence 5.1 Benchmark and Coverage 4 Single-hop Specific facts LoCoMo (Maharana et al., 2024) is the standard bench- mark for long-term conversational memory evaluation, 5 Adversarial Binary; excluded used by the related work we compare against. It com- prises 10 multi-session dialogues averaging 600 turns 5.4 Metrics and Models and 16 000 tokens across up to 32 sessions. We evaluate the text-only subset; locomo10.json contains 1 986 QA LLM-judge accuracy (primary): gold and model an- pairs, of which 1 935 remain after excluding image/video swers passed to Claude Haiku at temperature 0 (prompt questions with a keyword filter (51 excluded). in Appendix E). Validated on a 60-question blind human Of these 1 935 text-only questions, 1 501 are categories 1­ sample: 95% agreement (57/60), 100% on category 2 4 and form the primary aggregate (Table 1); the remaining (17/17), all 3 disagreements conservative (judge said NO 434 category 5 (adversarial) questions are excluded be- when human said YES). cause their binary scoring rule--awarding 1 if the model says `no information available'--rewards content removal Token F1 computed for consistency with prior work but and conflates memory quality with response calibration not used in the main analysis; values in public results (Appendix D). files. 1MacBook Air, Apple M-series chip. All results use temperature 0. A prior variance analysis at API default temperature (50 sampled pairs, 3 reruns) found mean SD 0.047; temperature-0 reruns shifted ag- gregates by 0.009. Bootstrap CIs: 95%, 2 000 resamples, seed 42, from the matched 1 935-question dataset (ci_report_10conv_matched.json); 1 501 ques- 3 Table 1: Aggregate judge accuracy, categories 1­4 (N = Table 4 shows per-conversation category 2 results. 1 501/condition). Full context (convs 0­1, 299 questions): 0.706 overall, 0.745 on cat. 2; performance ceiling only. 7 Discussion Condition Acc 95% CI Tok/Q 7.1 The Temporal Anchor Mechanism SWC-Temporal 0.468 [0.444, 0.495] 4 257 Three independent sources mutually confirm the mech- SWC-Full 0.379 [0.354, 0.404] 4 130 anism. The category accuracy pattern shows SWC-Full Sliding window 0.238 [0.216, 0.260] 4 125 fails specifically on temporal questions (0.156) while Truncation 0.171 [0.153, 0.190] 3 662 performing well on single-hop (0.459) and multi-hop (0.407) -- consistent with selective, not general, fail- tions for Table 1; category-specific counts for Table 2; ure. The preservation analysis quantifies the selectivity: n = 315 for the paired category-2 delta. temporal preservation jumps 20-fold (3.05%62.39%) while entities and events barely change. The ablation QA model: Claude Sonnet 4.6 (claude-sonnet-4-6). closes the argument: +0.314 judge accuracy (paired, n = 315) from one added sentence, with non-overlapping Compression/judge: Claude Haiku CIs against SWC-Full, without materially reducing multi- hop or single-hop performance. (claude-haiku-4-5-20251001). Embeddings: 7.2 Temporal, Not Episodic all-MiniLM-L6-v2. SWC-Full scores 0.459 on single-hop factual questions -- 6 Results specific facts from specific conversational turns, episodic in the conventional sense. The framework does not fail on 6.1 Aggregate Results episodic content generally; it fails on temporal informa- tion. The abstraction prompt covers facts, relationships, Table 1 presents results on the primary analysis set: cate- decisions, and plans -- none of these are temporal an- gories 1­4, 1 501 questions per condition. chors. Temporal anchors fall outside all protected cate- CIs are non-overlapping between SWC conditions and gories by default. SWC-Temporal corrects this precisely. baselines. SWC-Temporal achieves 0.468, 0.298 above truncation and 0.231 above sliding window. 7.3 Open-Domain Results 6.2 Results by Category Category 3 CIs overlap completely across all conditions. Commonsense inference questions depend substantially Table 2 presents judge accuracy by category. on model prior knowledge rather than compression strat- egy, making them less sensitive to what is preserved. No Paired delta, category 2: SWC-Temporal - SWC-Full conclusion can be drawn. = +0.314 [0.254, 0.375], n = 315. The direction is pos- itive in all ten conversations (per-conv deltas: +0.433, 7.4 Multi-hop and Single-hop Advantages +0.160, +0.259, +0.379, +0.385, +0.291, +0.177, +0.341, +0.364, +0.226; unweighted mean +0.301). Both SWC conditions have non-overlapping CIs with both baselines on categories 1 and 4. SWC conditions score On category 2, SWC-Temporal's CI [0.416, 0.524] does 0.407­0.429 on multi-hop against baselines of 0.157­ not overlap with SWC-Full [0.117, 0.197] or either base- 0.271, and 0.459­0.486 on single-hop against 0.170­ line. On categories 1 and 4, both SWC conditions have 0.238. These gains are measured over all ten conver- non-overlapping CIs with both baselines. Category 3 CIs sations and reflect the advantage of preserving relational overlap for all conditions; no reliable difference can be and biographical structure over simple truncation. concluded. 7.5 Conversation 1 Outlier 6.3 Preservation Analysis Sliding window (0.520) exceeds SWC-Temporal (0.440) Table 3 shows verbatim preservation rates across all ten on category 2 in conversation 1. The SWC-T vs SWC-F conversations. delta is still +0.160. Conversation 1 has only 25 tempo- Temporal protection causes an approximately 20-fold in- crease in temporal preservation while barely affecting named entity (×1.02) and event preservation (×1.11). The gist abstraction was always preserving events and par- ticipants; it was selectively omitting when they occurred. 4 Table 2: Judge accuracy by category, matched question set. Category 2 (temporal) highlighted. SWC-Temporal SWC-Full Sliding Truncation Category Acc 95% CI Acc 95% CI Acc 95% CI Acc 95% CI 1 Multi-hop 0.429 [0.371, 0.489] 0.407 [0.354, 0.464] 0.271 [0.221, 0.325] 0.157 [0.114, 0.200] 2 Temporal 0.470 [0.416, 0.524] 0.156 [0.117, 0.197] 0.171 [0.130, 0.213] 0.137 [0.102, 0.178] 3 Open-domain 0.427 [0.333, 0.521] 0.354 [0.260, 0.448] 0.354 [0.260, 0.458] 0.323 [0.229, 0.427] 4 Single-hop 0.486 [0.453, 0.517] 0.459 [0.425, 0.493] 0.238 [0.211, 0.267] 0.170 [0.144, 0.198] Table 3: Verbatim preservation rates by content type, all ten 7.7 Limitations conversations. Coverage: Matched intersection covers all ten conversa- Type Found SWC-F SWC-T × tions (1 935 text-only questions; primary aggregate uses 1 501 categories 1­4). Full context: Evaluated on conver- Temporal 952 3.05% 62.39% 20 sations 0­1 only; reference ceiling. Preservation metric: Named entity 7 559 8.03% 8.22% 1.02 Verbatim rate is a conservative proxy and may undercount Event 7 072 5.03% 5.61% 1.11 semantically faithful paraphrases. Transfer: Temporal protection tested only in SWC; whether the same fix im- Table 4: Category 2 (temporal) judge accuracy by conversa- proves sliding-window summarisation is left to future tion. Unweighted per-conv mean; matched paired estimate is work. Open-domain: Category 3 CIs overlap; no reliable +0.314 [0.254, 0.375] (Table 2). conclusion. Scheduling: Cannot be tested on an offline benchmark. C T F Sl Tr 8 Conclusion 0 0.595 0.162 +0.433 0.216 0.189 1 0.440 0.280 +0.160 0.520 0.320 Generic gist abstraction preserves relational, event, and 2 0.370 0.111 +0.259 0.074 0.111 factual content while selectively losing temporal anchors, 3 0.541 0.162 +0.379 0.135 0.081 because they are absent from standard prompt specifica- 4 0.615 0.231 +0.385 0.115 0.115 tions. A preservation analysis confirms this: only 3.05% 5 0.333 0.042 +0.291 0.208 0.208 of temporal expressions survive verbatim in generic sum- 6 0.265 0.088 +0.177 0.147 0.088 maries, compared to 8% for named entities and 5% 7 0.512 0.171 +0.341 0.195 0.171 for event mentions. 8 0.667 0.303 +0.364 0.061 0.111 9 0.290 0.065 +0.226 0.097 0.065 A one-sentence prompt modification recovers +0.314 [0.254, 0.375] judge accuracy on category-2 (temporal) Mean 0.463 0.162 +0.301 0.177 0.146 questions (matched set, direction confirmed in all ten conversations), without materially reducing multi-hop or ral questions and sliding window's 0.520 is anomalous single-hop performance. The temporal protection is a relative to its 0.061­0.216 range on the other nine conver- targeted fix, not a general compression relaxation. sations. Small cell sizes explain the deviation. Temporal anchors should be treated as a protected cate- 7.6 Implications gory in any gist compression component. Category-level evaluation paired with preservation rate analysis provides Any memory system using gist abstraction should ex- a more complete picture of what a memory strategy pre- plicitly protect temporal anchors. The fix is a prompt serves -- and what it leaves behind. modification, not an architectural change. Acknowledgements The category-level evaluation methodology and the preser- vation rate analysis together provide a more informa- The authors used Claude (Anthropic) for assistance with tive picture than aggregate accuracy alone. Systems that literature review, drafting, and experimental design. achieve high aggregate scores may mask systematic fail- ures on specific question types. 5 References memory system for personalized AI agents. arXiv preprint arXiv:2604.04853, 2026. P. Chhikara, D. Khant, S. Aryan, T. Singh, and D. Ya- dav. Mem0: Building production-ready AI agents Y. Xie. Learning to forget: Sleep-inspired memory con- with scalable long-term memory. arXiv preprint solidation for resolving proactive interference in large arXiv:2504.19413, 2025. language models. arXiv preprint arXiv:2603.14517, 2026. C. Cruz. Adaptive focus memory for language models. arXiv preprint arXiv:2511.12712, 2025. A Worked Gist Abstraction Example J. Fang, X. Deng, H. Xu, Z. Jiang, Y. Tang, Z. Xu, Original session: "I finally went to the LGBTQ support S. Deng, Y. Yao, M. Wang, S. Qiao, H. Chen, group last night -- 7 May. It was really moving. I spoke and N. Zhang. LightMem: Lightweight and effi- for about ten minutes about my experience coming out to cient memory-augmented generation. arXiv preprint my parents." arXiv:2510.18866, 2025. SWC-Full: "Caroline attended an LGBTQ support group and shared her coming-out experience, finding it emotion- M. Kang et al. ACON: Optimizing context compres- ally meaningful." sion for long-horizon LLM agents. arXiv preprint SWC-Temporal: "On 7 May, Caroline attended an arXiv:2510.00615, 2025. LGBTQ support group and spoke for ten minutes about her coming-out experience, finding it emotionally mean- N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, ingful." F. Petroni, and P. Liang. Lost in the middle: How SWC-Full loses the date (7 May) and duration (ten min- language models use long contexts. Transactions of utes). Named entities and the event are preserved in both; the Association for Computational Linguistics, 12:157­ only temporal anchors differ. 173, 2024. B Baseline Results Across All Ten Conversations A. Maharana, D.-H. Lee, S. Tulyakov, M. Bansal, F. Bar- bieri, and Y. Fang. Evaluating very long-term conver- Table 5: Overall judge accuracy, all ten conversations (includes sational memory of LLM agents. In Proceedings of the category 5; Table 1 uses categories 1­4 only). 62nd Annual Meeting of the Association for Computa- tional Linguistics, pages 13851­13870, 2024. Conv Q Trunc Sliding J. L. McClelland, B. L. McNaughton, and R. C. O'Reilly. 0 197 0.355 0.386 Why there are complementary learning systems in the 1 102 0.422 0.461 hippocampus and neocortex. Psychological Review, 2 191 0.298 0.330 102(3):419­457, 1995. 3 237 0.304 0.354 4 240 0.317 0.329 D. Patel and S. Patel. ENGRAM: Effective, lightweight 5 158 0.367 0.367 memory orchestration for conversational agents. arXiv 6 189 0.249 0.344 preprint arXiv:2511.12960, 2025. 7 233 0.283 0.343 8 192 0.297 0.328 G. Tononi and C. Cirelli. Sleep function and synaptic 9 196 0.260 0.316 homeostasis. Sleep Medicine Reviews, 10(1):49­62, 2006. Pooled 1 935 0.305 0.347 C. Wang and J. V. Sun. Unable to forget: Proactive in- Sliding window matches or exceeds truncation across all terference reveals working memory limits in LLMs be- ten conversations (conversation 5 is a tie at 0.367). yond context length. arXiv preprint arXiv:2506.08184, 2025. C SWC-Reactive Results S. Wang, E. Yu, O. Love, T. Zhang, T. Wong, S. Scargall, SWC-Reactive (token-threshold triggering only, API de- and C. Fan. MemMachine: A ground-truth-preserving fault temperature, conversations 0­2) achieved aggregate 6 judge accuracy of 0.452 vs SWC-Full's 0.462. Scheduling differences are not a validated contribution of this paper. D Category 5 (Adversarial) Results Category 5 questions are designed to be unanswerable; correct responses contain "no information available." Conditions removing more content score higher (trun- cation: 0.72­0.86 across conversations). This reflects content retention, not memory quality. E Judge Prompt and Validation Judge prompt (temperature 0): Question: {question}. Gold answer: {gold_answer}. Model answer: {model_answer}. Does the model answer correctly answer the question, given the gold answer? Reply only YES or NO. Validation: 60-question blind sample; 95% agreement (57/60); 100% on category 2 (17/17); all 3 disagreements conservative. Category examples: Cat. 1 (multi-hop): "What fields would Caroline likely pursue?" Cat. 2 (temporal): "When did Caroline go to the LGBTQ support group?" Cat. 3 (open-domain): "Would Caroline likely have Dr Seuss books?" Cat. 4 (single-hop): "What did the charity race raise awareness for?" Cat. 5 (adversarial): Unanswer- able questions; correct response acknowledges lack of information. 7