[RFC] Tail-Optimized LRU (T-LRU): Reducing Tail Latency via Conversation-Aware KV Cache Eviction
March 22, 2026 ยท #37823
Python
Difficulty: Easy
Labels
RFC
Parent Repository
vllm-project/vllm
Python repository
75,721 15,332