[Paper] MemT-1.0: Towards Memory-Native Evolution

[u/pmttyji]

Memory-native evolution requires making memory an intrinsic capability of foundation models, enabling them to continuously accumulate and utilize information. To this end, we introduce MemT-1.0, a new architecture of memory foundation models that internalize memory as part of model computation. Building on Metis, MemT-1.0 employs Gated DeltaNet-2 with a stable token-level memory update mechanism within each turn. We further introduce Mixture-of-Storage to scale native memory capacity. Following a deliberate backbone preparation, we develop a two-stage training pipeline consisting of continued pre-training and post-training, supported by 34B tokens of training data and efficient training and inference infrastructure. We release a model family comprising MemT-1.0-Nano and MemT-1.0-Flash, and conduct a comprehensive evaluation of memory and general capabilities. The results show that they outperform the parametric memory baselines on average and remain competitive with retrieval-augmented methods, while largely preserving general capabilities. At a history length of 128K tokens, MemT-1.0-Nano achieves approximately 1.86× end-to-end inference speedup and a 66× reduction in persistent storage relative to full context KV cache. By internalizing retrieved procedural experience, MemT-1.0 improves task success on agentic tasks, providing a practical step toward memory-native evolution.

• arXiv : https://arxiv.org/abs/2610.26825
• PDF : https://arxiv.org/pdf/2610.26825
• GitHub : https://github.com/MemTensor/MemT-1.0
• HuggingFace : https://huggingface.co/collections/MemTensor/memt-1

[u/Patentsmatter]

right now it's faster, but accuracy is still lacking in some areas.

[u/FireSilicon_]

Latency looks amazing. Might be good for real time applications.

[u/BenjiWithTheGoodHair]

The storage chart is the sleeper detail. Memory stays at 40 MiB whether you’re at 256K or 1M tokens, so you could save a small memory file per project instead of re-prefilling a huge context. The speedup also grows with length, from 1.86x at 128K to about 4x at 256K. Recall is still mixed against plain RAG though. Would love to see someone test Nano on a 1M token chat log.