I think this might be an interesting idea:
instead of passing the full chat to the LLM you pass only the recent messages, but give it the tools to read the previous messages, search through the chat and so on.
has something like this been tried?
pull down to refresh
I believe this is how most harnesses work already: most recent chat messages get sent as context then memory tool calls can look at earlier stuff