Memory and model routing

Before an agent runs, the pipeline loads its memory and picks a model. beam_memory and beam_routing provide both.

Memory

Memory is organized in scopes. A scope is a name and an id, such as session:5f89… or docset:d1. Each scope holds items of two kinds:

• Turns: the messages of a conversation. The pipeline saves every input and reply to the session scope.
• Facts: short texts with a kind, such as a rule or a preference. They load as system messages of the form [{kind} {scope}] {text}.

Loading layers

By default the pipeline loads only the session scope. Pass metadata.memory_scopes to load several layers, in order:

{
  "agent_name": "author",
  "session_id": "5f8915eb-31eb-4f8a-9517-e2c15ce7bc41",
  "content": [{"type": "text", "text": "Draft the release notes"}],
  "metadata": {
    "memory_scopes": [
      {"name": "system"},
      {"name": "organization", "id": "acme"},
      {"name": "session", "id": "5f8915eb-31eb-4f8a-9517-e2c15ce7bc41"}
    ]
  }
}

The loaded messages are trimmed to the resolved model's max_tokens. TokenLimitStrategy keeps the newest messages that fit, estimating about four characters per token.

When you pass memory_scopes, include the session scope yourself if you want the conversation history.

Do not let clients choose memory scopes

The pipeline loads whatever scopes the request names, without checking who is asking. A client that can set metadata.memory_scopes can read another session's history or another organization's facts. Remove the key from client requests and set the scopes in your API from the authenticated caller.

Storing facts

let id = beam.memory_remember("organization", "acme", "style", "Use short sentences.").await?;
beam.memory_forget(&id).await?;

Using the manager directly

use std::sync::Arc;

use beam_memory::{InMemoryBackend, MemoryManager, MemoryScope, TokenLimitStrategy};

let memory = MemoryManager::new(Arc::new(InMemoryBackend::new()), Arc::new(TokenLimitStrategy));
memory.remember(&MemoryScope::system(), "fact", "system rule").await?;
let context = memory
    .load_context(&[MemoryScope::system(), MemoryScope::session("s1")], 4096)
    .await?;
beam.set_memory(memory);

Type	Role
MemoryScope	new(name, id), system(), session(id)
MemoryBackend	Storage trait: load_layer, append_layer, forget. InMemoryBackend is the shipped implementation.
ContextStrategy	Trimming trait: apply(messages, max_tokens). TokenLimitStrategy is the shipped implementation.
MemoryManager	load_context, save_turn, remember, forget

Implement MemoryBackend to keep memory in your own storage.

Model routing

ModelRouter picks a ModelConfig for a request:

1. If the request's model_override names a known model, use it.
2. Otherwise run the strategies in order, each narrowing the candidates:
   • ModalityStrategy: keep models that support every modality in the request content.
   • CapabilityStrategy: keep models with the capabilities in metadata.capabilities, or, when that is absent, with the agent's tags that some model lists as a capability.
   • CostAwareStrategy: sort by cost_per_1k_tokens, cheapest first.
   • FallbackStrategy: return all models when nothing is left.
3. If no candidate remains, use default_model, or fail with BEM-041.
4. Otherwise return the first candidate.

use beam_routing::{load_models_from_yaml, ModelRouter};

let models = load_models_from_yaml("models.yaml")?;
let router = ModelRouter::new(models, Some("gpt-4o-mini".into()))?;
let model = router.resolve(&input, None)?;

ModelRouter::with_strategies(models, strategies, default_model) replaces the strategy list with your own RoutingStrategy implementations. Duplicate model names fail with BEM-042.

Model list file

load_models_from_yaml reads a models list of ModelConfig entries:

models.yaml

models:
  - name: gpt-4o-mini
    provider: openai          # openai, anthropic, custom, ollama, groq, nvidia
    modalities: [text]        # text, image, audio, video; default [text]
    capabilities: [tools]
    cost_per_1k_tokens: 0.15  # default 0.0
    max_tokens: 4096          # default 4096

provider_config and extra accept free-form JSON. Unknown keys are ignored. A read error is BEM-043 and a parse error is BEM-044. Keep API keys out of this file.

Current limitations

• RiverBeam builds its own router with one model, gpt-4o-mini, and has no method to replace it. ModelRouter and the YAML loader work on their own, but you cannot plug them into the pipeline yet.
• With the default strategies, FallbackStrategy never changes the result: the router stops as soon as the candidate list is empty, before the fallback runs.
• InMemoryBackend is the only memory backend; memory does not persist across restarts and is not shared between processes. The agent_memory table in the schema is not used by beam_memory yet.
• The worker tasks remember_memory and forget_memory return BEM-092. Call memory_remember and memory_forget on RiverBeam instead.