The challenges you're describing are exactly what Anthropic hit with Claude Code. Fragile retrieval, inconsistent results, the model not properly understanding context that was handed to it. Their solution? They dropped RAG entirely and let the model search the codebase itself via grep. It built a far better mental map that way. Wrote about their full process here: https://reading.sh/anthropic-revealed-how-they-build-claude-codes-brain-11e48e75fd01?sk=6662727c70ed637cd1692a81f33139e2
The challenges you're describing are exactly what Anthropic hit with Claude Code. Fragile retrieval, inconsistent results, the model not properly understanding context that was handed to it. Their solution? They dropped RAG entirely and let the model search the codebase itself via grep. It built a far better mental map that way. Wrote about their full process here: https://reading.sh/anthropic-revealed-how-they-build-claude-codes-brain-11e48e75fd01?sk=6662727c70ed637cd1692a81f33139e2