#RAG

MSA or memory sparse attention, a new AI system that can remember 100 million tokens with less than 9% accuracy loss, current AI models forget everything after about 128,000 tokens.