Building Context Compression in Astreus

Long conversations with AI agents create a fundamental problem. Every API call sends the entire message history, causing token costs to escalate and eventually hitting model limits. Astreus solves this with automatic context compression that summarizes older messages while maintaining conversational coherence. ## The Context Window Challenge Language models are stateless. They don't remember previous exchanges unless you include them in every request. A 50-message conversation can easily consume 25,000+ tokens per API call, creating escalating costs and slower response times. Even with 128K token limits in GPT-4o or 200K in Claude, production applications will eventually hit these ceilings. Without compression, you're forced to either truncate history (losing context) or restart conversations (breaking continuity). ## Enabling Auto Context Compression Astreus provides `autoContextCompression` as a configuration option that works alongside the memory system. When enabled, it automatically summarizes older messages during extended dialogues, preserving important details through intelligent summarization rather than truncation. Here's the real Astreus API for enabling context compression: ```javascript const agent = await Agent.create({ name: 'ContextAgent', model: 'gpt-4o', memory: true, autoContextCompression: true, systemPrompt: 'You can handle very long conversations efficiently.' }); ``` The `memory` option must be enabled for compression to work. This ensures conversational history is tracked and available for summarization. ## How It Works in Practice When your conversation grows beyond a certain threshold, Astreus automatically triggers compression. The system analyzes older messages and creates concise summaries that capture key information while drastically reducing token count. Recent messages remain in full detail for immediate context. This balances the need for rich recent context with the efficiency of summarized history. ```javascript // After 20+ exchanges, the agent still remembers early details const response = await agent.ask('What was the first space fact I mentioned?'); // The system retrieves information from compressed history console.log(response); // References the original fact despite many intervening messages ``` This demonstrates memory persistence across extended conversations. The agent maintains understanding of early exchanges even after dozens of subsequent messages. ## Environment Setup To use context compression, you need two environment variables configured: ```bash OPENAI_API_KEY=your_openai_api_key_here DB_URL=sqlite://./memory.db ``` The `DB_URL` is essential because memory and compression require persistent storage. SQLite works well for development, while production deployments typically use PostgreSQL or other robust databases. ## Testing Compression Effectiveness The best way to verify compression is through extended conversations. Send 20-30 messages on various topics, then ask the agent to recall early information. ```javascript // Start a long conversation for (let i = 0; i < 20; i++) { await agent.ask(`Tell me space fact number ${i + 1}`); } // Verify early context is preserved const recall = await agent.ask('What was space fact number 1?'); ``` If the agent accurately recalls information from early in the conversation, compression is working correctly. The memory persists despite the conversation length and automatic summarization. ## Cost Savings at Scale Without compression, a 100-message conversation at 500 tokens each sends 50,000 tokens per request. At $0.01 per 1K tokens, that's $0.50 per message just for input. With compression, you might send only 5,000 tokens (recent messages plus summary), dropping costs to $0.05 per message. That's 90% savings that compounds with every exchange. ```javascript // Rough token calculation const uncompressedTokens = messageCount * avgTokensPerMessage; const compressedTokens = (recentMessages * avgTokensPerMessage) + summaryTokens; const savings = ((uncompressedTokens - compressedTokens) / uncompressedTokens) * 100; console.log(`Token reduction: ${savings.toFixed(1)}%`); ``` For production applications handling thousands of conversations daily, these savings translate to thousands of dollars in reduced API costs. ## Getting Started You can explore context compression through two paths. Clone the official example repository from `astreus-ai/context-compression` for a complete working implementation, or install `@astreus-ai/astreus` directly via npm and integrate it into your existing project. The example repository includes a pre-configured setup that demonstrates compression in action with a simple chat interface. ```bash # Install Astreus npm install @astreus-ai/astreus # Or clone the example git clone https://github.com/astreus-ai/context-compression.git cd context-compression npm install ``` Once installed, set your environment variables and run the example to see compression working in real-time. ## When to Use Compression Context compression shines in customer support bots, personal assistants, and any application where conversations naturally extend beyond 10-15 exchanges. Short interactions (under 10 messages) rarely benefit from compression since they don't approach token limits. For applications where users return to conversations over days or weeks, compression becomes essential. It enables indefinite conversation length without degrading performance or hitting hard limits. ```javascript // Ideal for long-running assistants const personalAssistant = await Agent.create({ name: 'PersonalAssistant', model: 'gpt-4o', memory: true, autoContextCompression: true, systemPrompt: 'You are a helpful personal assistant who remembers past conversations.' }); // Supports multi-day conversations const morning = await personalAssistant.ask('What are my tasks today?'); // ... many hours and messages later ... const evening = await personalAssistant.ask('Did I complete everything we discussed this morning?'); ``` The agent maintains continuity across the entire day despite potentially hundreds of intervening messages. ## Best Practices Enable compression from the start if you anticipate conversations exceeding 10-15 exchanges. Retrofitting compression into existing systems with uncompressed history can be challenging. Monitor your actual token usage through OpenAI's dashboard or your database logs. This reveals whether compression is triggering appropriately and delivering expected savings. ```javascript // Example conversation with compression const response = await agent.ask('What tasks did we discuss earlier?'); console.log(response); ``` If you notice token counts staying consistently high despite many messages, verify that both `memory` and `autoContextCompression` are enabled. ## Compression vs. Memory Understand the distinction between these two features. Memory persists facts and context between sessions, while compression manages token efficiency within a single conversation thread. They work together synergistically. Memory stores what to remember, compression decides how to represent it efficiently. ```javascript // Both features enabled for optimal behavior const agent = await Agent.create({ name: 'OptimalAgent', model: 'gpt-4o', memory: true, // Long-term fact persistence autoContextCompression: true, // Efficient context management systemPrompt: 'Remember important details and handle long conversations.' }); ``` This configuration gives you both long-term memory across sessions and efficient handling of extended conversations within sessions. ## Production Considerations In production, ensure your database can handle the write load from memory operations. SQLite works for low-traffic applications, but high-volume deployments need PostgreSQL or similar robust solutions. Test compression with realistic conversation patterns. Run automated tests that simulate 50+ message exchanges and verify that critical information persists throughout. ```javascript // Automated compression test async function testCompression() { const agent = await Agent.create({ name: 'TestAgent', model: 'gpt-4o', memory: true, autoContextCompression: true }); // Send many messages await agent.ask('My favorite color is blue.'); for (let i = 0; i < 30; i++) { await agent.ask(`Tell me fact number ${i}`); } // Verify early info is retained const response = await agent.ask('What is my favorite color?'); assert(response.includes('blue'), 'Compression failed to preserve early context'); } ``` This automated test ensures compression doesn't lose critical details during summarization. ## The Results Context compression transforms AI agents from token-limited conversations into persistent assistants capable of indefinite dialogue. You gain predictable costs regardless of conversation length, consistent performance without degradation, and the ability to maintain natural long-running interactions. In production deployments, teams report 85-95% token cost reductions for conversations exceeding 20 messages. Users experience faster responses due to smaller context windows, and developers gain confidence that conversations won't fail when approaching model limits. For any Astreus application handling extended interactions, enabling `autoContextCompression` is a simple configuration change that delivers immediate benefits. This experiment is written for Astreus v0.5.37. Please ensure you are using a compatible version.