{"id":97841,"date":"2026-09-21T13:00:50","date_gmt":"2026-09-21T10:00:50","guid":{"rendered":"https:\/\/twodots.gr\/?p=97841"},"modified":"2026-09-21T13:00:51","modified_gmt":"2026-09-21T10:00:51","slug":"kvboost-taxytera-llm-koino-periexomeno-oxi-stin-arxi","status":"publish","type":"post","link":"https:\/\/twodots.gr\/bg\/kvboost-taxytera-llm-koino-periexomeno-oxi-stin-arxi\/","title":{"rendered":"KVBoost: \u03c4\u03b1\u03c7\u03cd\u03c4\u03b5\u03c1\u03b1 LLM \u03cc\u03c4\u03b1\u03bd \u03c4\u03bf \u03ba\u03bf\u03b9\u03bd\u03cc \u03c0\u03b5\u03c1\u03b9\u03b5\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf \u03b4\u03b5\u03bd \u03b2\u03c1\u03af\u03c3\u03ba\u03b5\u03c4\u03b1\u03b9 \u03c3\u03c4\u03b7\u03bd \u03b1\u03c1\u03c7\u03ae"},"content":{"rendered":"<div class=\"td-article-lede\">\n<p><strong>\u0391\u03c0\u03ac\u03bd\u03c4\u03b7\u03c3\u03b7 \u03c0\u03c1\u03ce\u03c4\u03b1:<\/strong> \u03c4\u03bf KVBoost \u03bc\u03b5\u03b9\u03ce\u03bd\u03b5\u03b9 \u03c4\u03bf\u03bd \u03c7\u03c1\u03cc\u03bd\u03bf \u03bc\u03ad\u03c7\u03c1\u03b9 \u03c4\u03bf \u03c0\u03c1\u03ce\u03c4\u03bf token \u03cc\u03c4\u03b1\u03bd \u03b5\u03c0\u03b1\u03bd\u03b1\u03bb\u03b1\u03bc\u03b2\u03b1\u03bd\u03cc\u03bc\u03b5\u03bd\u03bf \u03c0\u03b5\u03c1\u03b9\u03b5\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf \u03c5\u03c0\u03ac\u03c1\u03c7\u03b5\u03b9 \u03bc\u03ad\u03c3\u03b1 \u03c3\u03c4\u03bf prompt \u03b1\u03bb\u03bb\u03ac \u03cc\u03c7\u03b9 \u03b1\u03c0\u03b1\u03c1\u03b1\u03af\u03c4\u03b7\u03c4\u03b1 \u03c3\u03c4\u03b7\u03bd \u03b1\u03c1\u03c7\u03ae \u03c4\u03bf\u03c5. \u0391\u03bd\u03c4\u03af \u03bd\u03b1 \u03b6\u03b7\u03c4\u03ac \u03ba\u03bf\u03b9\u03bd\u03cc leading prefix, \u03c4\u03b5\u03bc\u03b1\u03c7\u03af\u03b6\u03b5\u03b9 \u03c4\u03b7\u03bd \u03b5\u03af\u03c3\u03bf\u03b4\u03bf, \u03b1\u03bd\u03b1\u03b6\u03b7\u03c4\u03ac \u03af\u03b4\u03b9\u03b1 chunks \u03ba\u03b1\u03b9 \u03b5\u03c0\u03b1\u03bd\u03b1\u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03b5\u03af \u03c4\u03b7\u03bd \u03b1\u03bd\u03c4\u03af\u03c3\u03c4\u03bf\u03b9\u03c7\u03b7 KV cache.<\/p>\n<p>\u03a3\u03c4\u03bf benchmark \u03c4\u03b7\u03c2 \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1\u03c2 \u03bc\u03b5 1.000 \u03b4\u03b5\u03af\u03b3\u03bc\u03b1\u03c4\u03b1 bug localization, Qwen2.5-3B \u03ba\u03b1\u03b9 \u03bc\u03af\u03b1 RTX 4060, \u03bf \u03bc\u03ad\u03c3\u03bf\u03c2 TTFT \u03ad\u03c0\u03b5\u03c3\u03b5 \u03c3\u03c4\u03b1 142,4 ms \u03b1\u03c0\u03cc 639,1 ms \u03b3\u03b9\u03b1 full recomputation \u03ba\u03b1\u03b9 165,5 ms \u03b3\u03b9\u03b1 vLLM prefix caching. \u03a4\u03bf \u03ba\u03ad\u03c1\u03b4\u03bf\u03c2 \u03b4\u03b5\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03b4\u03c9\u03c1\u03b5\u03ac\u03bd: chunks \u03b1\u03c0\u03cc \u03ac\u03bb\u03bb\u03b7 \u03b8\u03ad\u03c3\u03b7 \u03ae \u03ac\u03bb\u03bb\u03bf \u03c0\u03c1\u03bf\u03b7\u03b3\u03bf\u03cd\u03bc\u03b5\u03bd\u03bf context \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 seam repair, \u03b1\u03c5\u03c3\u03c4\u03b7\u03c1\u03ae \u03bc\u03ad\u03c4\u03c1\u03b7\u03c3\u03b7 \u03c0\u03bf\u03b9\u03cc\u03c4\u03b7\u03c4\u03b1\u03c2 \u03ba\u03b1\u03b9 fallback \u03c0\u03c1\u03b9\u03bd \u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03b7\u03b8\u03bf\u03cd\u03bd \u03c3\u03b5 production LLM, RAG \u03ae AI-agent \u03c1\u03bf\u03ad\u03c2.<\/p>\n<\/div>\n<div class=\"td-article-toc\">\n<div class=\"td-toc-title\">\u0421\u044a\u0434\u044a\u0440\u0436\u0430\u043d\u0438\u0435<\/div>\n<ul>\n<li><a href=\"#ti-einai-kvboost\">\u03a4\u03b9 \u03b5\u03af\u03bd\u03b1\u03b9 \u03c4\u03bf KVBoost \u03ba\u03b1\u03b9 \u03c0\u03bf\u03b9\u03bf \u03ba\u03b5\u03bd\u03cc \u03ba\u03b1\u03bb\u03cd\u03c0\u03c4\u03b5\u03b9<\/a><\/li>\n<li><a href=\"#orio-prefix-caching\">\u0393\u03b9\u03b1\u03c4\u03af \u03c4\u03bf prefix caching \u03c7\u03ac\u03bd\u03b5\u03b9 \u03c7\u03c1\u03ae\u03c3\u03b9\u03bc\u03b7 \u03b5\u03c0\u03b1\u03bd\u03ac\u03bb\u03b7\u03c8\u03b7<\/a><\/li>\n<li><a href=\"#chunks-dual-hash\">Chunks \u03ba\u03b1\u03b9 dual hash: exact \u03ba\u03b1\u03b9 approximate hits<\/a><\/li>\n<li><a href=\"#seam-error-rope\">Seam error \u03ba\u03b1\u03b9 RoPE: \u03b3\u03b9\u03b1\u03c4\u03af \u03b7 \u03bc\u03b5\u03c4\u03b1\u03c6\u03bf\u03c1\u03ac \u03b4\u03b5\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03b1\u03ba\u03c1\u03b9\u03b2\u03ae\u03c2<\/a><\/li>\n<li><a href=\"#selective-cacheblend\">SelectiveRecompute \u03ba\u03b1\u03b9 CacheBlendRecompute<\/a><\/li>\n<li><a href=\"#memory-quantization\">Memory budget, eviction \u03ba\u03b1\u03b9 quantization<\/a><\/li>\n<li><a href=\"#benchmark-setup\">\u03a4\u03b9 \u03b1\u03ba\u03c1\u03b9\u03b2\u03ce\u03c2 \u03bc\u03b5\u03c4\u03c1\u03ae\u03b8\u03b7\u03ba\u03b5 \u03c3\u03c4\u03bf benchmark<\/a><\/li>\n<li><a href=\"#apotelesmata-ttft\">\u03a4\u03b9 \u03b4\u03b5\u03af\u03c7\u03bd\u03bf\u03c5\u03bd \u03c4\u03b1 \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03ad\u03c3\u03bc\u03b1\u03c4\u03b1 latency \u03ba\u03b1\u03b9 \u03c0\u03bf\u03b9\u03cc\u03c4\u03b7\u03c4\u03b1\u03c2<\/a><\/li>\n<li><a href=\"#pote-kerdizei\">\u03a0\u03cc\u03c4\u03b5 \u03ba\u03b5\u03c1\u03b4\u03af\u03b6\u03b5\u03b9 \u2014 \u03ba\u03b1\u03b9 \u03c0\u03cc\u03c4\u03b5 \u03cc\u03c7\u03b9<\/a><\/li>\n<li><a href=\"#symvatotita-periorismoi\">\u03a3\u03c5\u03bc\u03b2\u03b1\u03c4\u03cc\u03c4\u03b7\u03c4\u03b1 \u03ba\u03b1\u03b9 \u03c4\u03b5\u03c7\u03bd\u03b9\u03ba\u03bf\u03af \u03c0\u03b5\u03c1\u03b9\u03bf\u03c1\u03b9\u03c3\u03bc\u03bf\u03af<\/a><\/li>\n<li><a href=\"#rag-agents-proionta\">\u03a4\u03b9 \u03c3\u03b7\u03bc\u03b1\u03af\u03bd\u03b5\u03b9 \u03b3\u03b9\u03b1 RAG, agents \u03ba\u03b1\u03b9 AI \u03c0\u03c1\u03bf\u03ca\u03cc\u03bd\u03c4\u03b1<\/a><\/li>\n<li><a href=\"#production-risk\">\u0391\u03c0\u03cc \u03c4\u03bf benchmark \u03c3\u03b5 \u03b5\u03bb\u03b5\u03b3\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf production rollout<\/a><\/li>\n<li><a href=\"#plaisio-apofasis\">\u0395\u03c0\u03c4\u03ac \u03ad\u03bb\u03b5\u03b3\u03c7\u03bf\u03b9 \u03c0\u03c1\u03b9\u03bd \u03b1\u03c0\u03cc \u03c4\u03b7\u03bd \u03b1\u03c0\u03cc\u03c6\u03b1\u03c3\u03b7<\/a><\/li>\n<\/ul>\n<\/div>\n<h2 id=\"ti-einai-kvboost\">\u03a4\u03b9 \u03b5\u03af\u03bd\u03b1\u03b9 \u03c4\u03bf KVBoost \u03ba\u03b1\u03b9 \u03c0\u03bf\u03b9\u03bf \u03ba\u03b5\u03bd\u03cc \u03ba\u03b1\u03bb\u03cd\u03c0\u03c4\u03b5\u03b9<\/h2>\n<p>\u039a\u03b1\u03c4\u03ac \u03c4\u03bf prefill, \u03ad\u03bd\u03b1 decoder-only transformer \u03b5\u03c0\u03b5\u03be\u03b5\u03c1\u03b3\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03c4\u03bf prompt \u03c0\u03c1\u03b9\u03bd \u03be\u03b5\u03ba\u03b9\u03bd\u03ae\u03c3\u03b5\u03b9 \u03c4\u03bf autoregressive decode. \u0393\u03b9\u03b1 \u03ba\u03ac\u03b8\u03b5 token \u03b4\u03b7\u03bc\u03b9\u03bf\u03c5\u03c1\u03b3\u03b5\u03af key \u03ba\u03b1\u03b9 value tensors \u03c0\u03bf\u03c5 \u03ba\u03c1\u03b1\u03c4\u03bf\u03cd\u03bd \u03c4\u03b7\u03bd \u03c0\u03bb\u03b7\u03c1\u03bf\u03c6\u03bf\u03c1\u03af\u03b1 \u03c0\u03c1\u03bf\u03c3\u03bf\u03c7\u03ae\u03c2. \u038c\u03c4\u03b1\u03bd \u03ad\u03bd\u03b1 \u03bc\u03b5\u03b3\u03ac\u03bb\u03bf system prompt, \u03ad\u03bd\u03b1 retrieved document \u03ae \u03ad\u03bd\u03b1 few-shot example \u03b5\u03bc\u03c6\u03b1\u03bd\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 \u03be\u03b1\u03bd\u03ac, \u03bf \u03c0\u03bb\u03ae\u03c1\u03b7\u03c2 \u03b5\u03c0\u03b1\u03bd\u03c5\u03c0\u03bf\u03bb\u03bf\u03b3\u03b9\u03c3\u03bc\u03cc\u03c2 \u03b1\u03c5\u03c4\u03ce\u03bd \u03c4\u03c9\u03bd tensors \u03c0\u03c1\u03bf\u03c3\u03b8\u03ad\u03c4\u03b5\u03b9 latency \u03c0\u03c1\u03b9\u03bd \u03b5\u03bc\u03c6\u03b1\u03bd\u03b9\u03c3\u03c4\u03b5\u03af \u03b7 \u03b1\u03c0\u03ac\u03bd\u03c4\u03b7\u03c3\u03b7.<\/p>\n<p>\u03a4\u03bf KVBoost \u03b5\u03af\u03bd\u03b1\u03b9 \u03b5\u03c1\u03b5\u03c5\u03bd\u03b7\u03c4\u03b9\u03ba\u03cc \u03c3\u03cd\u03c3\u03c4\u03b7\u03bc\u03b1 chunk-level KV cache reuse \u03c4\u03bf\u03c5 Srihari Unnikrishnan \u03b3\u03b9\u03b1 Hugging Face decoder models \u03bc\u03b5 RoPE. \u0397 \u03b2\u03b1\u03c3\u03b9\u03ba\u03ae \u03c4\u03bf\u03c5 \u03c5\u03c0\u03cc\u03b8\u03b5\u03c3\u03b7 \u03b5\u03af\u03bd\u03b1\u03b9 \u03cc\u03c4\u03b9 \u03c4\u03bf \u03ba\u03bf\u03b9\u03bd\u03cc \u03c0\u03b5\u03c1\u03b9\u03b5\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf \u03b4\u03b5\u03bd \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03bd\u03b1 \u03b2\u03c1\u03af\u03c3\u03ba\u03b5\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc \u03c4\u03b7 \u03b8\u03ad\u03c3\u03b7 \u03bc\u03b7\u03b4\u03ad\u03bd \u03b3\u03b9\u03b1 \u03bd\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 \u03c5\u03c0\u03bf\u03bb\u03bf\u03b3\u03b9\u03c3\u03c4\u03b9\u03ba\u03ac \u03c7\u03c1\u03ae\u03c3\u03b9\u03bc\u03bf. \u0388\u03bd\u03b1 document chunk \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b5\u03bc\u03c6\u03b1\u03bd\u03b9\u03c3\u03c4\u03b5\u03af \u03bc\u03b5\u03c4\u03ac \u03b1\u03c0\u03cc \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03b5\u03c4\u03b9\u03ba\u03cc preamble, \u03b1\u03bd\u03ac\u03bc\u03b5\u03c3\u03b1 \u03c3\u03b5 \u03b5\u03be\u03b1\u03c4\u03bf\u03bc\u03b9\u03ba\u03b5\u03c5\u03bc\u03ad\u03bd\u03b5\u03c2 \u03bf\u03b4\u03b7\u03b3\u03af\u03b5\u03c2 \u03ae \u03c3\u03b5 \u03ac\u03bb\u03bb\u03b7 \u03c3\u03b5\u03b9\u03c1\u03ac \u03bc\u03ad\u03c3\u03b1 \u03c3\u03b5 \u03ad\u03bd\u03b1 RAG prompt.<\/p>\n<p>\u0391\u03c5\u03c4\u03ae \u03b7 \u03c5\u03c0\u03cc\u03b8\u03b5\u03c3\u03b7 \u03b1\u03c6\u03bf\u03c1\u03ac \u03b5\u03b9\u03b4\u03b9\u03ba\u03ac \u03c4\u03bf prefill. \u0394\u03b5\u03bd \u03ba\u03ac\u03bd\u03b5\u03b9 \u03c4\u03bf autoregressive decode \u03c4\u03b1\u03c7\u03cd\u03c4\u03b5\u03c1\u03bf \u03b1\u03c0\u03cc \u03bc\u03cc\u03bd\u03b7 \u03c4\u03b7\u03c2. \u0397 \u03b5\u03c0\u03af\u03c3\u03b7\u03bc\u03b7 \u03c4\u03b5\u03ba\u03bc\u03b7\u03c1\u03af\u03c9\u03c3\u03b7 \u03c4\u03bf\u03c5 vLLM \u03b5\u03c0\u03b9\u03c3\u03b7\u03bc\u03b1\u03af\u03bd\u03b5\u03b9 \u03c4\u03bf \u03af\u03b4\u03b9\u03bf \u03cc\u03c1\u03b9\u03bf \u03b3\u03b9\u03b1 \u03c4\u03bf Automatic Prefix Caching: \u03c4\u03bf \u03cc\u03c6\u03b5\u03bb\u03bf\u03c2 \u03bc\u03b5\u03b9\u03ce\u03bd\u03b5\u03c4\u03b1\u03b9 \u03cc\u03c4\u03b1\u03bd \u03bf \u03c0\u03b5\u03c1\u03b9\u03c3\u03c3\u03cc\u03c4\u03b5\u03c1\u03bf\u03c2 \u03c7\u03c1\u03cc\u03bd\u03bf\u03c2 \u03ba\u03b1\u03c4\u03b1\u03bd\u03b1\u03bb\u03ce\u03bd\u03b5\u03c4\u03b1\u03b9 \u03c3\u03c4\u03b7 \u03b4\u03b7\u03bc\u03b9\u03bf\u03c5\u03c1\u03b3\u03af\u03b1 \u03c0\u03bf\u03bb\u03bb\u03ce\u03bd \u03bd\u03ad\u03c9\u03bd tokens \u03ba\u03b1\u03b9 \u03cc\u03c7\u03b9 \u03c3\u03c4\u03b7\u03bd \u03b5\u03c0\u03b5\u03be\u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1 \u03c4\u03b7\u03c2 \u03b5\u03b9\u03c3\u03cc\u03b4\u03bf\u03c5.<\/p>\n<p>\u0397 \u03b4\u03b9\u03ac\u03ba\u03c1\u03b9\u03c3\u03b7 \u03b5\u03af\u03bd\u03b1\u03b9 \u03c3\u03b7\u03bc\u03b1\u03bd\u03c4\u03b9\u03ba\u03ae \u03b3\u03b9\u03b1 \u03ba\u03ac\u03b8\u03b5 <a href=\"https:\/\/twodots.gr\/vllm-transformers-native-speed-ai-inference\/\">LLM inference stack \u03c0\u03bf\u03c5 \u03c3\u03c5\u03bd\u03b4\u03c5\u03ac\u03b6\u03b5\u03b9 vLLM \u03ba\u03b1\u03b9 Transformers<\/a>. \u03a4\u03bf \u03b5\u03c1\u03ce\u03c4\u03b7\u03bc\u03b1 \u03b4\u03b5\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03b1\u03c0\u03bb\u03ce\u03c2 \u00ab\u03c5\u03c0\u03ac\u03c1\u03c7\u03b5\u03b9 cache;\u00bb, \u03b1\u03bb\u03bb\u03ac \u03c0\u03bf\u03b9\u03bf \u03c4\u03bc\u03ae\u03bc\u03b1 \u03c4\u03b7\u03c2 latency \u03b4\u03b9\u03b1\u03b4\u03c1\u03bf\u03bc\u03ae\u03c2 \u03ba\u03b1\u03bb\u03cd\u03c0\u03c4\u03b5\u03b9, \u03c3\u03b5 \u03c0\u03bf\u03b9\u03b1 prompts \u03b2\u03c1\u03af\u03c3\u03ba\u03b5\u03b9 hits \u03ba\u03b1\u03b9 \u03c4\u03b9 \u03ba\u03cc\u03c3\u03c4\u03bf\u03c2 \u03b5\u03b9\u03c3\u03ac\u03b3\u03b5\u03b9 \u03b7 \u03b5\u03c0\u03b1\u03bd\u03b1\u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b7.<\/p>\n<h2 id=\"orio-prefix-caching\">\u0393\u03b9\u03b1\u03c4\u03af \u03c4\u03bf prefix caching \u03c7\u03ac\u03bd\u03b5\u03b9 \u03c7\u03c1\u03ae\u03c3\u03b9\u03bc\u03b7 \u03b5\u03c0\u03b1\u03bd\u03ac\u03bb\u03b7\u03c8\u03b7<\/h2>\n<p>\u03a4\u03bf \u03ba\u03bb\u03b1\u03c3\u03b9\u03ba\u03cc prefix cache \u03b5\u03c0\u03b1\u03bd\u03b1\u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03b5\u03af KV blocks \u03cc\u03c4\u03b1\u03bd \u03ad\u03bd\u03b1 \u03bd\u03ad\u03bf \u03b1\u03af\u03c4\u03b7\u03bc\u03b1 \u03be\u03b5\u03ba\u03b9\u03bd\u03ac \u03bc\u03b5 \u03c4\u03b7\u03bd \u03af\u03b4\u03b9\u03b1 \u03b1\u03ba\u03bf\u03bb\u03bf\u03c5\u03b8\u03af\u03b1 tokens \u03bc\u03b5 \u03c0\u03b1\u03bb\u03b1\u03b9\u03cc\u03c4\u03b5\u03c1\u03bf \u03b1\u03af\u03c4\u03b7\u03bc\u03b1. \u0397 \u03c0\u03c1\u03bf\u03c3\u03ad\u03b3\u03b3\u03b9\u03c3\u03b7 \u03b5\u03af\u03bd\u03b1\u03b9 \u03b9\u03c3\u03c7\u03c5\u03c1\u03ae \u03b3\u03b9\u03b1 \u03c3\u03c4\u03b1\u03b8\u03b5\u03c1\u03ac system prompts, \u03b5\u03c0\u03b1\u03bd\u03b1\u03bb\u03b1\u03bc\u03b2\u03b1\u03bd\u03cc\u03bc\u03b5\u03bd\u03b5\u03c2 \u03b5\u03c1\u03c9\u03c4\u03ae\u03c3\u03b5\u03b9\u03c2 \u03c0\u03ac\u03bd\u03c9 \u03c3\u03c4\u03bf \u03af\u03b4\u03b9\u03bf \u03bc\u03b1\u03ba\u03c1\u03cd \u03ad\u03b3\u03b3\u03c1\u03b1\u03c6\u03bf \u03ae \u03b4\u03b9\u03b1\u03b4\u03bf\u03c7\u03b9\u03ba\u03bf\u03cd\u03c2 \u03b3\u03cd\u03c1\u03bf\u03c5\u03c2 \u03c4\u03b7\u03c2 \u03af\u03b4\u03b9\u03b1\u03c2 \u03c3\u03c5\u03bd\u03bf\u03bc\u03b9\u03bb\u03af\u03b1\u03c2. \u038c\u03c4\u03b1\u03bd \u03cc\u03bc\u03c9\u03c2 \u03b1\u03bb\u03bb\u03ac\u03be\u03b5\u03b9 \u03c4\u03bf \u03c0\u03c1\u03ce\u03c4\u03bf \u03bc\u03ad\u03c1\u03bf\u03c2, \u03bf \u03ba\u03bf\u03b9\u03bd\u03cc\u03c2 \u03ba\u03bf\u03c1\u03bc\u03cc\u03c2 \u03c0\u03bf\u03c5 \u03b1\u03ba\u03bf\u03bb\u03bf\u03c5\u03b8\u03b5\u03af \u03b4\u03b5\u03bd \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03b5\u03af \u03c0\u03bb\u03ad\u03bf\u03bd \u03ba\u03bf\u03b9\u03bd\u03cc prefix.<\/p>\n<p>\u03a3\u03b5 \u03ad\u03bd\u03b1 RAG pipeline, \u03c4\u03b1 \u03af\u03b4\u03b9\u03b1 documents \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b1\u03bd\u03b1\u03b4\u03b9\u03b1\u03c4\u03ac\u03c3\u03c3\u03bf\u03bd\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc \u03c4\u03bf\u03bd retriever, \u03bd\u03b1 \u03c0\u03c1\u03bf\u03b7\u03b3\u03b5\u03af\u03c4\u03b1\u03b9 \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03b5\u03c4\u03b9\u03ba\u03ae \u03bf\u03b4\u03b7\u03b3\u03af\u03b1 \u03ae \u03bd\u03b1 \u03c0\u03b1\u03c1\u03b5\u03bc\u03b2\u03ac\u03bb\u03bb\u03b5\u03c4\u03b1\u03b9 user-specific context. \u03a3\u03b5 \u03ad\u03bd\u03b1\u03bd agent, tool descriptions \u03ba\u03b1\u03b9 \u03c0\u03bf\u03bb\u03b9\u03c4\u03b9\u03ba\u03ad\u03c2 \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b5\u03c0\u03b1\u03bd\u03b5\u03bc\u03c6\u03b1\u03bd\u03af\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 \u03b1\u03bd\u03ac\u03bc\u03b5\u03c3\u03b1 \u03c3\u03b5 \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03b5\u03c4\u03b9\u03ba\u03ac \u03b2\u03ae\u03bc\u03b1\u03c4\u03b1. \u03a4\u03bf \u03b1\u03c0\u03bf\u03c4\u03ad\u03bb\u03b5\u03c3\u03bc\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03ae \u03b5\u03c0\u03b1\u03bd\u03ac\u03bb\u03b7\u03c8\u03b7 \u03c3\u03c4\u03bf \u03c0\u03b5\u03c1\u03b9\u03b5\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf, \u03b1\u03bb\u03bb\u03ac \u03bc\u03b7\u03b4\u03b5\u03bd\u03b9\u03ba\u03cc leading-prefix match.<\/p>\n<p>\u03a4\u03bf KVBoost \u03b1\u03bb\u03bb\u03ac\u03b6\u03b5\u03b9 \u03c4\u03b7 \u03bc\u03bf\u03bd\u03ac\u03b4\u03b1 \u03c4\u03b1\u03c5\u03c4\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b7\u03c2 \u03b1\u03c0\u03cc \u03bf\u03bb\u03cc\u03ba\u03bb\u03b7\u03c1\u03b7 \u03c4\u03b7\u03bd \u03ba\u03bf\u03b9\u03bd\u03ae \u03b1\u03c1\u03c7\u03ae \u03c3\u03b5 token chunks. \u039c\u03b5 \u03c0\u03c1\u03bf\u03b5\u03c0\u03b9\u03bb\u03b5\u03b3\u03bc\u03ad\u03bd\u03bf \u03bc\u03ad\u03b3\u03b5\u03b8\u03bf\u03c2 128 tokens, \u03c8\u03ac\u03c7\u03bd\u03b5\u03b9 \u03b5\u03c0\u03b1\u03bd\u03b1\u03bb\u03b1\u03bc\u03b2\u03b1\u03bd\u03cc\u03bc\u03b5\u03bd\u03b1 \u03c4\u03bc\u03ae\u03bc\u03b1\u03c4\u03b1 \u03c3\u03b5 \u03bf\u03c0\u03bf\u03b9\u03b1\u03b4\u03ae\u03c0\u03bf\u03c4\u03b5 \u03b8\u03ad\u03c3\u03b7 \u03c4\u03b7\u03c2 \u03b5\u03b9\u03c3\u03cc\u03b4\u03bf\u03c5. \u03a4\u03bf \u03c0\u03bb\u03b5\u03bf\u03bd\u03ad\u03ba\u03c4\u03b7\u03bc\u03b1 \u03c3\u03c5\u03bd\u03bf\u03b4\u03b5\u03cd\u03b5\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc \u03bd\u03ad\u03bf \u03c1\u03af\u03c3\u03ba\u03bf: \u03b7 \u03ba\u03b1\u03c4\u03ac\u03c3\u03c4\u03b1\u03c3\u03b7 \u03b5\u03bd\u03cc\u03c2 transformer token \u03b5\u03be\u03b1\u03c1\u03c4\u03ac\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc \u03cc,\u03c4\u03b9 \u03c0\u03c1\u03bf\u03b7\u03b3\u03ae\u03b8\u03b7\u03ba\u03b5, \u03ac\u03c1\u03b1 \u03c4\u03bf \u03af\u03b4\u03b9\u03bf \u03ba\u03b5\u03af\u03bc\u03b5\u03bd\u03bf \u03c3\u03b5 \u03ac\u03bb\u03bb\u03b7 \u03b8\u03ad\u03c3\u03b7 \u03ae \u03bc\u03b5\u03c4\u03ac \u03b1\u03c0\u03cc \u03ac\u03bb\u03bb\u03bf context \u03b4\u03b5\u03bd \u03ad\u03c7\u03b5\u03b9 \u03ba\u03b1\u03c4\u2019 \u03b1\u03bd\u03ac\u03b3\u03ba\u03b7 \u03af\u03b4\u03b9\u03b1 KV tensors.<\/p>\n<div class=\"td-comparison td-comparison-cards td-comparison-cards--horizontal\">\n<div class=\"td-comparison-grid td-comparison-grid--three\">\n<div class=\"td-platform-card\">\n<p class=\"td-comparison-title\">Full recomputation<\/p>\n<p>\u03a5\u03c0\u03bf\u03bb\u03bf\u03b3\u03af\u03b6\u03b5\u03b9 \u03be\u03b1\u03bd\u03ac \u03bf\u03bb\u03cc\u03ba\u03bb\u03b7\u03c1\u03bf \u03c4\u03bf prompt. \u0395\u03af\u03bd\u03b1\u03b9 \u03c4\u03bf \u03b1\u03c3\u03c6\u03b1\u03bb\u03ad\u03c2 baseline \u03b3\u03b9\u03b1 \u03c0\u03bf\u03b9\u03cc\u03c4\u03b7\u03c4\u03b1, \u03b1\u03bb\u03bb\u03ac \u03c0\u03bb\u03b7\u03c1\u03ce\u03bd\u03b5\u03b9 \u03cc\u03bb\u03bf \u03c4\u03bf prefill latency \u03b1\u03ba\u03cc\u03bc\u03b7 \u03ba\u03b1\u03b9 \u03cc\u03c4\u03b1\u03bd \u03bc\u03b5\u03b3\u03ac\u03bb\u03bf \u03bc\u03ad\u03c1\u03bf\u03c2 \u03c4\u03b7\u03c2 \u03b5\u03b9\u03c3\u03cc\u03b4\u03bf\u03c5 \u03b5\u03c0\u03b1\u03bd\u03b1\u03bb\u03b1\u03bc\u03b2\u03ac\u03bd\u03b5\u03c4\u03b1\u03b9.<\/p>\n<div class=\"td-badge-row\"><span class=\"td-badge\">\u0391\u03ba\u03c1\u03b9\u03b2\u03ad\u03c2<\/span><span class=\"td-badge\">\u03a5\u03c8\u03b7\u03bb\u03cc prefill<\/span><\/div>\n<\/div>\n<div class=\"td-platform-card\">\n<p class=\"td-comparison-title\">vLLM prefix cache<\/p>\n<p>\u0395\u03c0\u03b1\u03bd\u03b1\u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03b5\u03af \u03ba\u03bf\u03b9\u03bd\u03ac \u03b1\u03c1\u03c7\u03b9\u03ba\u03ac blocks \u03c7\u03c9\u03c1\u03af\u03c2 seam repair. \u0395\u03af\u03bd\u03b1\u03b9 \u03b9\u03b4\u03b1\u03bd\u03b9\u03ba\u03cc \u03cc\u03c4\u03b1\u03bd \u03c4\u03b1 prompts \u03ad\u03c7\u03bf\u03c5\u03bd \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc \u03ba\u03bf\u03b9\u03bd\u03cc prefix, \u03b1\u03bb\u03bb\u03ac \u03b4\u03b5\u03bd \u03b1\u03be\u03b9\u03bf\u03c0\u03bf\u03b9\u03b5\u03af \u03bc\u03b5\u03c4\u03b1\u03ba\u03b9\u03bd\u03b7\u03bc\u03ad\u03bd\u03b1 chunks.<\/p>\n<div class=\"td-badge-row\"><span class=\"td-badge\">Leading prefix<\/span><span class=\"td-badge\">\u03a7\u03c9\u03c1\u03af\u03c2 approximate hit<\/span><\/div>\n<\/div>\n<div class=\"td-platform-card td-platform-card--navy\">\n<p class=\"td-comparison-title\">KVBoost<\/p>\n<p>\u0391\u03bd\u03b1\u03b6\u03b7\u03c4\u03ac \u03af\u03b4\u03b9\u03bf \u03c0\u03b5\u03c1\u03b9\u03b5\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf \u03c3\u03b5 chunk \u03b5\u03c0\u03af\u03c0\u03b5\u03b4\u03bf. \u03a4\u03b1 exact hits \u03b5\u03c0\u03b1\u03bd\u03b1\u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03bf\u03cd\u03bd\u03c4\u03b1\u03b9 \u03ac\u03bc\u03b5\u03c3\u03b1, \u03b5\u03bd\u03ce \u03c4\u03b1 content-only hits \u03b1\u03c0\u03b1\u03b9\u03c4\u03bf\u03cd\u03bd deviation-guided repair.<\/p>\n<div class=\"td-badge-row\"><span class=\"td-badge\">\u039f\u03c0\u03bf\u03b9\u03b1\u03b4\u03ae\u03c0\u03bf\u03c4\u03b5 \u03b8\u03ad\u03c3\u03b7<\/span><span class=\"td-badge\">Seam repair<\/span><\/div>\n<\/div>\n<\/div>\n<\/div>\n<h2 id=\"chunks-dual-hash\">Chunks \u03ba\u03b1\u03b9 dual hash: exact \u03ba\u03b1\u03b9 approximate hits<\/h2>\n<p>\u03a4\u03bf ChunkRegistry \u03c7\u03c9\u03c1\u03af\u03b6\u03b5\u03b9 \u03c4\u03b7\u03bd \u03b5\u03af\u03c3\u03bf\u03b4\u03bf \u03c3\u03b5 \u03c3\u03c4\u03b1\u03b8\u03b5\u03c1\u03ac chunks, \u03bc\u03b5 \u03c0\u03c1\u03bf\u03b5\u03c0\u03b9\u03bb\u03bf\u03b3\u03ae 128 tokens. \u0397 fixed \u03c3\u03c4\u03c1\u03b1\u03c4\u03b7\u03b3\u03b9\u03ba\u03ae \u03ba\u03cc\u03b2\u03b5\u03b9 \u03c3\u03b5 \u03b1\u03ba\u03c1\u03b9\u03b2\u03ae offsets, \u03b7 semantic \u03c3\u03c4\u03c1\u03b1\u03c4\u03b7\u03b3\u03b9\u03ba\u03ae \u03bc\u03b5\u03c4\u03b1\u03ba\u03b9\u03bd\u03b5\u03af \u03c4\u03bf \u03cc\u03c1\u03b9\u03bf \u03c0\u03c1\u03bf\u03c2 \u03ba\u03bf\u03bd\u03c4\u03b9\u03bd\u03ae \u03c0\u03c1\u03cc\u03c4\u03b1\u03c3\u03b7 \u03ae \u03c0\u03b1\u03c1\u03ac\u03b3\u03c1\u03b1\u03c6\u03bf \u03ba\u03b1\u03b9 \u03b7 document \u03c3\u03c4\u03c1\u03b1\u03c4\u03b7\u03b3\u03b9\u03ba\u03ae \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b1\u03bd\u03c4\u03b9\u03bc\u03b5\u03c4\u03c9\u03c0\u03af\u03c3\u03b5\u03b9 \u03bf\u03bb\u03cc\u03ba\u03bb\u03b7\u03c1\u03bf \u03c4\u03bf input \u03c9\u03c2 \u03b5\u03bd\u03b9\u03b1\u03af\u03bf chunk \u03b3\u03b9\u03b1 \u03c0\u03c1\u03bf\u03b8\u03ad\u03c1\u03bc\u03b1\u03bd\u03c3\u03b7 \u03c4\u03b7\u03c2 cache.<\/p>\n<p>\u039a\u03ac\u03b8\u03b5 chunk \u03c0\u03b1\u03af\u03c1\u03bd\u03b5\u03b9 \u03b4\u03cd\u03bf \u03c4\u03b1\u03c5\u03c4\u03cc\u03c4\u03b7\u03c4\u03b5\u03c2. \u03a4\u03bf prefix hash \u03b1\u03bb\u03c5\u03c3\u03b9\u03b4\u03ce\u03bd\u03b5\u03b9 \u03c4\u03b1 tokens \u03c4\u03bf\u03c5 chunk \u03bc\u03b5 \u03c4\u03bf hash \u03c4\u03bf\u03c5 \u03c0\u03c1\u03bf\u03b7\u03b3\u03bf\u03cd\u03bc\u03b5\u03bd\u03bf\u03c5 context. \u038a\u03b4\u03b9\u03bf prefix hash \u03c3\u03b7\u03bc\u03b1\u03af\u03bd\u03b5\u03b9 \u03af\u03b4\u03b9\u03b1 tokens \u03ba\u03b1\u03b9 \u03af\u03b4\u03b9\u03b1 \u03b9\u03c3\u03c4\u03bf\u03c1\u03af\u03b1 \u03c0\u03c1\u03b9\u03bd \u03b1\u03c0\u03cc \u03b1\u03c5\u03c4\u03ac, \u03b5\u03c0\u03bf\u03bc\u03ad\u03bd\u03c9\u03c2 \u03c4\u03bf hit \u03b8\u03b5\u03c9\u03c1\u03b5\u03af\u03c4\u03b1\u03b9 exact. \u03a4\u03bf content hash \u03c0\u03b5\u03c1\u03b9\u03bb\u03b1\u03bc\u03b2\u03ac\u03bd\u03b5\u03b9 \u03bc\u03cc\u03bd\u03bf \u03c4\u03b1 token IDs \u03c4\u03bf\u03c5 chunk \u03ba\u03b1\u03b9 \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b5\u03bd\u03c4\u03bf\u03c0\u03af\u03c3\u03b5\u03b9 \u03c4\u03bf \u03af\u03b4\u03b9\u03bf \u03c0\u03b5\u03c1\u03b9\u03b5\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf \u03b1\u03bd\u03b5\u03be\u03ac\u03c1\u03c4\u03b7\u03c4\u03b1 \u03b1\u03c0\u03cc \u03c4\u03b7 \u03b8\u03ad\u03c3\u03b7 \u03c4\u03bf\u03c5.<\/p>\n<p>\u0397 lookup cascade \u03b4\u03bf\u03ba\u03b9\u03bc\u03ac\u03b6\u03b5\u03b9 \u03c0\u03c1\u03ce\u03c4\u03b1 \u03c4\u03bf prefix hash, \u03bc\u03b5\u03c4\u03ac \u03c4\u03bf content hash \u03ba\u03b1\u03b9 \u03c4\u03ad\u03bb\u03bf\u03c2 \u03b4\u03b7\u03bb\u03ce\u03bd\u03b5\u03b9 miss. \u03a4\u03bf content-hash hit \u03b4\u03b5\u03bd \u03b2\u03b1\u03c6\u03c4\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 \u03b1\u03ba\u03c1\u03b9\u03b2\u03ad\u03c2. \u03a7\u03b1\u03c1\u03b1\u03ba\u03c4\u03b7\u03c1\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 approximate \u03b5\u03c0\u03b5\u03b9\u03b4\u03ae \u03c4\u03b1 cached keys \u03ad\u03c7\u03bf\u03c5\u03bd \u03c0\u03b1\u03c1\u03b1\u03c7\u03b8\u03b5\u03af \u03bc\u03b5 \u03ac\u03bb\u03bb\u03b7 RoPE \u03b8\u03ad\u03c3\u03b7 \u03ba\u03b1\u03b9 \u03b5\u03c0\u03b5\u03b9\u03b4\u03ae \u03c4\u03bf chunk \u03ad\u03c7\u03b5\u03b9 \u03b4\u03b5\u03b9 \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03b5\u03c4\u03b9\u03ba\u03cc \u03c0\u03c1\u03bf\u03b7\u03b3\u03bf\u03cd\u03bc\u03b5\u03bd\u03bf context. \u0391\u03c5\u03c4\u03cc\u03c2 \u03bf \u03b4\u03b9\u03b1\u03c7\u03c9\u03c1\u03b9\u03c3\u03bc\u03cc\u03c2 exact \u03ba\u03b1\u03b9 approximate \u03b5\u03af\u03bd\u03b1\u03b9 \u03b7 \u03bf\u03c5\u03c3\u03af\u03b1 \u03c4\u03bf\u03c5 \u03c3\u03c7\u03b5\u03b4\u03b9\u03b1\u03c3\u03bc\u03bf\u03cd, \u03cc\u03c7\u03b9 \u03b1\u03c0\u03bb\u03ce\u03c2 \u03bc\u03b9\u03b1 \u03b2\u03b5\u03bb\u03c4\u03b9\u03c3\u03c4\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b7 hash table.<\/p>\n<p>\u0397 \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1 \u03c0\u03b5\u03c1\u03b9\u03b3\u03c1\u03ac\u03c6\u03b5\u03b9 \u03b5\u03c0\u03af\u03c3\u03b7\u03c2 overlap tokens \u03ba\u03b1\u03b9 attention-sink tokens \u03c0\u03bf\u03c5 \u03bc\u03c0\u03bf\u03c1\u03bf\u03cd\u03bd \u03bd\u03b1 \u03b5\u03b9\u03c3\u03b1\u03c7\u03b8\u03bf\u03cd\u03bd \u03b3\u03b9\u03b1 \u03ba\u03b1\u03bb\u03cd\u03c4\u03b5\u03c1\u03b7 \u03c3\u03c5\u03bd\u03ad\u03c7\u03b5\u03b9\u03b1 \u03c3\u03c4\u03b1 \u03cc\u03c1\u03b9\u03b1. \u039f\u03b9 \u03c0\u03c1\u03bf\u03b5\u03c0\u03b9\u03bb\u03bf\u03b3\u03ad\u03c2 \u03c4\u03bf\u03c5\u03c2 \u03b5\u03af\u03bd\u03b1\u03b9 \u03bc\u03b7\u03b4\u03ad\u03bd. \u0394\u03b5\u03bd \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bb\u03bf\u03b9\u03c0\u03cc\u03bd \u03bd\u03b1 \u03c0\u03b1\u03c1\u03bf\u03c5\u03c3\u03b9\u03ac\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 \u03c9\u03c2 \u03c5\u03c0\u03bf\u03c7\u03c1\u03b5\u03c9\u03c4\u03b9\u03ba\u03ac \u03b5\u03bd\u03b5\u03c1\u03b3\u03ac \u03bf\u03cd\u03c4\u03b5 \u03c9\u03c2 \u03b4\u03c9\u03c1\u03b5\u03ac\u03bd: \u03b1\u03c5\u03be\u03ac\u03bd\u03bf\u03c5\u03bd \u03c4\u03bf live compute \u03b3\u03b9\u03b1 \u03bd\u03b1 \u03bc\u03b5\u03b9\u03ce\u03c3\u03bf\u03c5\u03bd \u03c0\u03b9\u03b8\u03b1\u03bd\u03ad\u03c2 boundary \u03b1\u03c0\u03bf\u03ba\u03bb\u03af\u03c3\u03b5\u03b9\u03c2.<\/p>\n<h2 id=\"seam-error-rope\">Seam error \u03ba\u03b1\u03b9 RoPE: \u03b3\u03b9\u03b1\u03c4\u03af \u03b7 \u03bc\u03b5\u03c4\u03b1\u03c6\u03bf\u03c1\u03ac \u03b4\u03b5\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03b1\u03ba\u03c1\u03b9\u03b2\u03ae\u03c2<\/h2>\n<p>\u038c\u03c4\u03b1\u03bd \u03b4\u03cd\u03bf independently cached chunks \u03b5\u03c0\u03b1\u03bd\u03b1\u03c3\u03c5\u03bd\u03b1\u03c1\u03bc\u03bf\u03bb\u03bf\u03b3\u03bf\u03cd\u03bd\u03c4\u03b1\u03b9, \u03c4\u03b1 tokens \u03ba\u03bf\u03bd\u03c4\u03ac \u03c3\u03c4\u03bf \u03c3\u03b7\u03bc\u03b5\u03af\u03bf \u03ad\u03bd\u03c9\u03c3\u03b7\u03c2 \u03b4\u03b5\u03bd \u03ad\u03c7\u03bf\u03c5\u03bd \u03c5\u03c0\u03bf\u03bb\u03bf\u03b3\u03b9\u03c3\u03c4\u03b5\u03af \u03bc\u03b5 \u03c4\u03bf \u03bd\u03ad\u03bf \u03b3\u03b5\u03b9\u03c4\u03bf\u03bd\u03b9\u03ba\u03cc context. \u0391\u03c5\u03c4\u03cc \u03b5\u03af\u03bd\u03b1\u03b9 \u03c4\u03bf seam error. \u0397 \u03b5\u03c0\u03af\u03b4\u03c1\u03b1\u03c3\u03b7 \u03b4\u03b5\u03bd \u03c0\u03b5\u03c1\u03b9\u03bf\u03c1\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 \u03ba\u03b1\u03c4\u2019 \u03b1\u03bd\u03ac\u03b3\u03ba\u03b7 \u03c3\u03c4\u03bf \u03c0\u03c1\u03ce\u03c4\u03bf token \u03c4\u03bf\u03c5 chunk, \u03b5\u03c0\u03b5\u03b9\u03b4\u03ae \u03bf\u03b9 attention \u03b1\u03bd\u03b1\u03c0\u03b1\u03c1\u03b1\u03c3\u03c4\u03ac\u03c3\u03b5\u03b9\u03c2 \u03bc\u03b5\u03c4\u03b1\u03c6\u03ad\u03c1\u03bf\u03c5\u03bd \u03c0\u03bb\u03b7\u03c1\u03bf\u03c6\u03bf\u03c1\u03af\u03b1 \u03bc\u03ad\u03c3\u03b1 \u03b1\u03c0\u03cc \u03b4\u03b9\u03b1\u03b4\u03bf\u03c7\u03b9\u03ba\u03ac layers.<\/p>\n<p>\u03a5\u03c0\u03ac\u03c1\u03c7\u03b5\u03b9 \u03ba\u03b1\u03b9 positional \u03c0\u03c1\u03cc\u03b2\u03bb\u03b7\u03bc\u03b1. \u03a4\u03b1 RoPE keys \u03b5\u03bd\u03c3\u03c9\u03bc\u03b1\u03c4\u03ce\u03bd\u03bf\u03c5\u03bd \u03c0\u03b5\u03c1\u03b9\u03c3\u03c4\u03c1\u03bf\u03c6\u03ad\u03c2 \u03c0\u03bf\u03c5 \u03c3\u03c5\u03bd\u03b4\u03ad\u03bf\u03bd\u03c4\u03b1\u03b9 \u03bc\u03b5 \u03c4\u03b7\u03bd \u03b1\u03c0\u03cc\u03bb\u03c5\u03c4\u03b7 \u03b8\u03ad\u03c3\u03b7. \u0388\u03bd\u03b1 chunk \u03c0\u03bf\u03c5 \u03b1\u03c0\u03bf\u03b8\u03b7\u03ba\u03b5\u03cd\u03c4\u03b7\u03ba\u03b5 \u03c3\u03c4\u03b9\u03c2 \u03b8\u03ad\u03c3\u03b5\u03b9\u03c2 0\u2013127 \u03b4\u03b5\u03bd \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03c4\u03bf\u03c0\u03bf\u03b8\u03b5\u03c4\u03b7\u03b8\u03b5\u03af \u03b1\u03c6\u03b5\u03bb\u03ce\u03c2 \u03c3\u03c4\u03b9\u03c2 \u03b8\u03ad\u03c3\u03b5\u03b9\u03c2 1.000\u20131.127 \u03ba\u03b1\u03b9 \u03bd\u03b1 \u03b8\u03b5\u03c9\u03c1\u03b7\u03b8\u03b5\u03af \u03b9\u03c3\u03bf\u03b4\u03cd\u03bd\u03b1\u03bc\u03bf. \u03a4\u03bf prompt assembler \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03b4\u03b9\u03b1\u03c4\u03b7\u03c1\u03b5\u03af \u03c4\u03b9\u03c2 \u03bd\u03ad\u03b5\u03c2 absolute position IDs, \u03bd\u03b1 \u03be\u03ad\u03c1\u03b5\u03b9 \u03c0\u03bf\u03b9\u03b1 chunks \u03b5\u03af\u03bd\u03b1\u03b9 exact \u03ae approximate \u03ba\u03b1\u03b9 \u03bd\u03b1 \u03ba\u03b1\u03c4\u03b1\u03b3\u03c1\u03ac\u03c6\u03b5\u03b9 \u03c4\u03b9\u03c2 \u03b8\u03ad\u03c3\u03b5\u03b9\u03c2 \u03c4\u03c9\u03bd seams.<\/p>\n<p>\u0391\u03c5\u03c4\u03cc \u03b5\u03be\u03b7\u03b3\u03b5\u03af \u03b3\u03b9\u03b1\u03c4\u03af \u03ad\u03bd\u03b1 content-addressed cache \u03c7\u03c9\u03c1\u03af\u03c2 repair \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 \u03b3\u03c1\u03ae\u03b3\u03bf\u03c1\u03bf \u03b1\u03bb\u03bb\u03ac \u03bb\u03b1\u03bd\u03b8\u03b1\u03c3\u03bc\u03ad\u03bd\u03bf. \u0397 \u03c4\u03b1\u03c5\u03c4\u03cc\u03c4\u03b7\u03c4\u03b1 \u03c4\u03bf\u03c5 \u03ba\u03b5\u03b9\u03bc\u03ad\u03bd\u03bf\u03c5 \u03b4\u03b5\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03af\u03b4\u03b9\u03b1 \u03bc\u03b5 \u03c4\u03b7\u03bd \u03c4\u03b1\u03c5\u03c4\u03cc\u03c4\u03b7\u03c4\u03b1 \u03c4\u03b7\u03c2 \u03ba\u03b1\u03c4\u03ac\u03c3\u03c4\u03b1\u03c3\u03b7\u03c2 \u03c4\u03bf\u03c5 transformer. \u03a4\u03bf KVBoost \u03c0\u03c1\u03bf\u03c3\u03c0\u03b1\u03b8\u03b5\u03af \u03bd\u03b1 \u03b1\u03bd\u03b1\u03ba\u03c4\u03ae\u03c3\u03b5\u03b9 \u03c4\u03bf \u03c5\u03c0\u03bf\u03bb\u03bf\u03b3\u03b9\u03c3\u03c4\u03b9\u03ba\u03cc \u03ba\u03ad\u03c1\u03b4\u03bf\u03c2 \u03c7\u03c9\u03c1\u03af\u03c2 \u03bd\u03b1 \u03b1\u03b3\u03bd\u03bf\u03ae\u03c3\u03b5\u03b9 \u03b1\u03c5\u03c4\u03ae \u03c4\u03b7 \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03ac.<\/p>\n<h2 id=\"selective-cacheblend\">SelectiveRecompute \u03ba\u03b1\u03b9 CacheBlendRecompute<\/h2>\n<p>\u03a4\u03bf SelectiveRecompute \u03b5\u03c0\u03b1\u03bd\u03b5\u03ba\u03c4\u03b5\u03bb\u03b5\u03af \u03ad\u03bd\u03b1 \u03c3\u03c4\u03b1\u03b8\u03b5\u03c1\u03cc \u03c0\u03b1\u03c1\u03ac\u03b8\u03c5\u03c1\u03bf \u03c3\u03c4\u03bf \u03c4\u03ad\u03bb\u03bf\u03c2 \u03ba\u03ac\u03b8\u03b5 cached chunk. \u0397 \u03c0\u03c1\u03bf\u03b5\u03c0\u03b9\u03bb\u03bf\u03b3\u03ae \u03c0\u03bf\u03c5 \u03c0\u03b5\u03c1\u03b9\u03b3\u03c1\u03ac\u03c6\u03b5\u03c4\u03b1\u03b9 \u03c3\u03c4\u03b7\u03bd \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 16 tokens. \u039c\u03b5 \u03b4\u03cd\u03bf seams \u03c3\u03b5 prompt 512 tokens, \u03c4\u03bf paper \u03b5\u03ba\u03c4\u03b9\u03bc\u03ac \u03cc\u03c4\u03b9 \u03b7 \u03c3\u03c4\u03c1\u03b1\u03c4\u03b7\u03b3\u03b9\u03ba\u03ae \u03b1\u03bd\u03c4\u03b9\u03c3\u03c4\u03bf\u03b9\u03c7\u03b5\u03af \u03c0\u03b5\u03c1\u03af\u03c0\u03bf\u03c5 \u03c3\u03c4\u03bf 8% \u03c4\u03bf\u03c5 \u03c0\u03bb\u03ae\u03c1\u03bf\u03c5\u03c2 prefill. \u0395\u03af\u03bd\u03b1\u03b9 \u03c0\u03c1\u03bf\u03b2\u03bb\u03ad\u03c8\u03b9\u03bc\u03b7 \u03ba\u03b1\u03b9 \u03b5\u03cd\u03ba\u03bf\u03bb\u03b7 \u03bd\u03b1 \u03b5\u03bb\u03b5\u03b3\u03c7\u03b8\u03b5\u03af, \u03b1\u03bb\u03bb\u03ac \u03b5\u03c0\u03b9\u03bb\u03ad\u03b3\u03b5\u03b9 tokens \u03bc\u03b5 \u03b2\u03ac\u03c3\u03b7 \u03c4\u03b7 \u03b8\u03ad\u03c3\u03b7 \u03ba\u03b1\u03b9 \u03cc\u03c7\u03b9 \u03c4\u03b7 \u03bc\u03b5\u03c4\u03c1\u03b7\u03bc\u03ad\u03bd\u03b7 \u03b1\u03c0\u03cc\u03ba\u03bb\u03b9\u03c3\u03b7.<\/p>\n<p>\u03a4\u03bf CacheBlendRecompute \u03ba\u03ac\u03bd\u03b5\u03b9 probe pass, \u03c3\u03c5\u03b3\u03ba\u03c1\u03af\u03bd\u03b5\u03b9 \u03c4\u03b1 \u03c0\u03b1\u03bb\u03b9\u03ac \u03bc\u03b5 \u03c4\u03b1 \u03b5\u03bd\u03b7\u03bc\u03b5\u03c1\u03c9\u03bc\u03ad\u03bd\u03b1 key tensors \u03bc\u03ad\u03c3\u03c9 cosine deviation \u03ba\u03b1\u03b9 \u03b5\u03c0\u03b1\u03bd\u03c5\u03c0\u03bf\u03bb\u03bf\u03b3\u03af\u03b6\u03b5\u03b9 \u03c4\u03b1 tokens \u03bc\u03b5 \u03c4\u03b7 \u03bc\u03b5\u03b3\u03b1\u03bb\u03cd\u03c4\u03b5\u03c1\u03b7 \u03b1\u03bb\u03bb\u03b1\u03b3\u03ae. \u0397 \u03c0\u03c1\u03bf\u03b5\u03c0\u03b9\u03bb\u03b5\u03b3\u03bc\u03ad\u03bd\u03b7 \u03c0\u03b5\u03c1\u03b9\u03b3\u03c1\u03b1\u03c6\u03ae \u03c3\u03c4\u03bf\u03c7\u03b5\u03cd\u03b5\u03b9 \u03c0\u03b5\u03c1\u03af\u03c0\u03bf\u03c5 \u03c3\u03c4\u03bf 15% \u03c4\u03bf\u03c5 prompt. \u0393\u03b9\u03b1 content-hash matches \u03b7 repair \u03b4\u03b9\u03b1\u03b4\u03b9\u03ba\u03b1\u03c3\u03af\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 \u03c5\u03c0\u03bf\u03c7\u03c1\u03b5\u03c9\u03c4\u03b9\u03ba\u03ae, \u03b5\u03c0\u03b5\u03b9\u03b4\u03ae \u03b1\u03c5\u03c4\u03ac \u03c4\u03b1 hits \u03ad\u03c7\u03bf\u03c5\u03bd \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03b5\u03c4\u03b9\u03ba\u03ae positional \u03ba\u03b1\u03b9 contextual \u03c0\u03c1\u03bf\u03ad\u03bb\u03b5\u03c5\u03c3\u03b7.<\/p>\n<p>\u0397 \u03b4\u03b5\u03cd\u03c4\u03b5\u03c1\u03b7 \u03c3\u03c4\u03c1\u03b1\u03c4\u03b7\u03b3\u03b9\u03ba\u03ae \u03b5\u03af\u03bd\u03b1\u03b9 \u03c0\u03b9\u03bf \u03c3\u03c4\u03bf\u03c7\u03b5\u03c5\u03bc\u03ad\u03bd\u03b7, \u03cc\u03bc\u03c9\u03c2 \u03c4\u03bf probe \u03ad\u03c7\u03b5\u03b9 \u03b4\u03b9\u03ba\u03cc \u03c4\u03bf\u03c5 latency. \u03a3\u03b5 \u03c0\u03bf\u03bb\u03cd \u03bc\u03b5\u03b3\u03ac\u03bb\u03bf cached context \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03ba\u03bf\u03c3\u03c4\u03af\u03c3\u03b5\u03b9 \u03b5\u03ba\u03b1\u03c4\u03bf\u03bd\u03c4\u03ac\u03b4\u03b5\u03c2 milliseconds \u03ba\u03b1\u03b9 \u03bd\u03b1 \u03ba\u03b1\u03c4\u03b1\u03bd\u03b1\u03bb\u03ce\u03c3\u03b5\u03b9 \u03bc\u03ad\u03c1\u03bf\u03c2 \u03c4\u03bf\u03c5 \u03b8\u03b5\u03c9\u03c1\u03b7\u03c4\u03b9\u03ba\u03bf\u03cd \u03ba\u03ad\u03c1\u03b4\u03bf\u03c5\u03c2. \u0393\u03b9\u2019 \u03b1\u03c5\u03c4\u03cc \u03c4\u03bf recompute ratio \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03bc\u03b5\u03c4\u03c1\u03b9\u03ad\u03c4\u03b1\u03b9 \u03bc\u03b1\u03b6\u03af \u03bc\u03b5 \u03c4\u03bf end-to-end TTFT \u03ba\u03b1\u03b9 \u03cc\u03c7\u03b9 \u03bd\u03b1 \u03b1\u03bd\u03c4\u03b9\u03bc\u03b5\u03c4\u03c9\u03c0\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 \u03c9\u03c2 \u03b1\u03c0\u03bf\u03bc\u03bf\u03bd\u03c9\u03bc\u03ad\u03bd\u03b7 \u03c1\u03cd\u03b8\u03bc\u03b9\u03c3\u03b7.<\/p>\n<aside class=\"td-article-note\"><strong>\u03a4\u03b9 \u03c4\u03b5\u03ba\u03bc\u03b7\u03c1\u03b9\u03ce\u03bd\u03b5\u03c4\u03b1\u03b9:<\/strong> \u03b7 \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1 \u03b4\u03b5\u03af\u03c7\u03bd\u03b5\u03b9 \u03cc\u03c4\u03b9 deviation-guided repair \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b4\u03b9\u03b1\u03c4\u03b7\u03c1\u03ae\u03c3\u03b5\u03b9 \u03c4\u03bf output agreement \u03c3\u03c4\u03bf \u03c3\u03c5\u03b3\u03ba\u03b5\u03ba\u03c1\u03b9\u03bc\u03ad\u03bd\u03bf Qwen2.5-3B bug-localization benchmark. \u0394\u03b5\u03bd \u03b1\u03c0\u03bf\u03b4\u03b5\u03b9\u03ba\u03bd\u03cd\u03b5\u03b9 \u03cc\u03c4\u03b9 \u03c4\u03bf \u03af\u03b4\u03b9\u03bf recompute ratio \u03b5\u03af\u03bd\u03b1\u03b9 \u03b1\u03c3\u03c6\u03b1\u03bb\u03ad\u03c2 \u03b3\u03b9\u03b1 long-form generation, code completion \u03ae \u03ba\u03ac\u03b8\u03b5 production \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03bf.<\/aside>\n<h2 id=\"memory-quantization\">Memory budget, eviction \u03ba\u03b1\u03b9 quantization<\/h2>\n<p>\u0397 cache \u03bb\u03b5\u03b9\u03c4\u03bf\u03c5\u03c1\u03b3\u03b5\u03af \u03ba\u03ac\u03c4\u03c9 \u03b1\u03c0\u03cc \u03c3\u03ba\u03bb\u03b7\u03c1\u03cc \u03cc\u03c1\u03b9\u03bf bytes \u03ce\u03c3\u03c4\u03b5 \u03bd\u03b1 \u03bc\u03b7 \u03b4\u03b9\u03b5\u03ba\u03b4\u03b9\u03ba\u03b5\u03af \u03b1\u03bd\u03b5\u03be\u03ad\u03bb\u03b5\u03b3\u03ba\u03c4\u03b1 \u03c4\u03b7 VRAM \u03c0\u03bf\u03c5 \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 \u03c4\u03b1 weights \u03ba\u03b1\u03b9 \u03c4\u03bf generation KV cache. \u03a0\u03c1\u03cc\u03c3\u03c6\u03b1\u03c4\u03b1 chunks \u03bc\u03c0\u03bf\u03c1\u03bf\u03cd\u03bd \u03bd\u03b1 \u03bc\u03ad\u03bd\u03bf\u03c5\u03bd pinned. \u03a4\u03b1 \u03c5\u03c0\u03cc\u03bb\u03bf\u03b9\u03c0\u03b1 \u03b2\u03b1\u03b8\u03bc\u03bf\u03bb\u03bf\u03b3\u03bf\u03cd\u03bd\u03c4\u03b1\u03b9 \u03bc\u03b5 proxy \u03c3\u03b7\u03bc\u03b1\u03bd\u03c4\u03b9\u03ba\u03cc\u03c4\u03b7\u03c4\u03b1\u03c2 \u03c0\u03bf\u03c5 \u03b2\u03b1\u03c3\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 \u03c3\u03c4\u03bf\u03bd \u03bc\u03ad\u03c3\u03bf L2 norm \u03c4\u03c9\u03bd key tensors, \u03b5\u03bd\u03ce \u03c4\u03bf LRU \u03bb\u03cd\u03bd\u03b5\u03b9 \u03b9\u03c3\u03bf\u03b2\u03b1\u03b8\u03bc\u03af\u03b5\u03c2 \u03ba\u03b1\u03c4\u03ac \u03c4\u03b7\u03bd \u03b1\u03c0\u03bf\u03bc\u03ac\u03ba\u03c1\u03c5\u03bd\u03c3\u03b7.<\/p>\n<p>\u03a0\u03c1\u03bf\u03b1\u03b9\u03c1\u03b5\u03c4\u03b9\u03ba\u03cc disk tier \u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03b5\u03af memory-mapped \u03b1\u03c1\u03c7\u03b5\u03af\u03bf \u03bc\u03b5 \u03c3\u03c4\u03b1\u03b8\u03b5\u03c1\u03ac slots \u03ba\u03b1\u03b9 index \u03b1\u03c0\u03cc hash \u03c3\u03b5 \u03b8\u03ad\u03c3\u03b7. \u0397 \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1 \u03b1\u03bd\u03b1\u03c6\u03ad\u03c1\u03b5\u03b9 10\u201350 ms \u03b3\u03b9\u03b1 \u03b1\u03bd\u03ac\u03ba\u03c4\u03b7\u03c3\u03b7 chunk \u03b1\u03c0\u03cc \u03b4\u03af\u03c3\u03ba\u03bf \u03ad\u03bd\u03b1\u03bd\u03c4\u03b9 100\u2013500 ms \u03b3\u03b9\u03b1 GPU recomputation. \u0391\u03c5\u03c4\u03ac \u03b5\u03af\u03bd\u03b1\u03b9 \u03bc\u03b5\u03b3\u03ad\u03b8\u03b7 \u03c4\u03b7\u03c2 \u03c0\u03b1\u03c1\u03bf\u03c5\u03c3\u03b9\u03b1\u03b6\u03cc\u03bc\u03b5\u03bd\u03b7\u03c2 \u03c5\u03bb\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b7\u03c2: storage, PCIe, tensor size, serialization \u03ba\u03b1\u03b9 concurrency \u03bc\u03c0\u03bf\u03c1\u03bf\u03cd\u03bd \u03bd\u03b1 \u03b1\u03bb\u03bb\u03ac\u03be\u03bf\u03c5\u03bd \u03c1\u03b9\u03b6\u03b9\u03ba\u03ac \u03c4\u03bf \u03b1\u03c0\u03bf\u03c4\u03ad\u03bb\u03b5\u03c3\u03bc\u03b1.<\/p>\n<p>\u0393\u03b9\u03b1 \u03bc\u03b5\u03b3\u03b1\u03bb\u03cd\u03c4\u03b5\u03c1\u03b7 \u03c7\u03c9\u03c1\u03b7\u03c4\u03b9\u03ba\u03cc\u03c4\u03b7\u03c4\u03b1, \u03c4\u03bf KVBoost \u03c5\u03bb\u03bf\u03c0\u03bf\u03b9\u03b5\u03af KIVI-style \u03b1\u03c3\u03cd\u03bc\u03bc\u03b5\u03c4\u03c1\u03bf quantization. \u03a4\u03b1 keys \u03ba\u03b2\u03b1\u03bd\u03c4\u03af\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 \u03b1\u03bd\u03ac channel \u03ba\u03b1\u03b9 \u03c4\u03b1 values \u03b1\u03bd\u03ac token, \u03bc\u03b5 int8 \u03ba\u03b1\u03b9 int4 \u03b5\u03c0\u03b9\u03bb\u03bf\u03b3\u03ad\u03c2. \u0397 \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1 \u03c0\u03b5\u03c1\u03b9\u03b3\u03c1\u03ac\u03c6\u03b5\u03b9 \u03ba\u03b1\u03c4\u03ac \u03c0\u03c1\u03bf\u03c3\u03ad\u03b3\u03b3\u03b9\u03c3\u03b7 \u03bc\u03b5\u03af\u03c9\u03c3\u03b7 \u03bc\u03bd\u03ae\u03bc\u03b7\u03c2 2\u00d7 \u03ba\u03b1\u03b9 4\u00d7 \u03b3\u03b9\u03b1 \u03c4\u03b1 cached tensors, \u03cc\u03c7\u03b9 \u03b1\u03bd\u03c4\u03af\u03c3\u03c4\u03bf\u03b9\u03c7\u03b7 \u03b5\u03b3\u03b3\u03c5\u03b7\u03bc\u03ad\u03bd\u03b7 \u03bc\u03b5\u03af\u03c9\u03c3\u03b7 \u03c4\u03b7\u03c2 \u03c3\u03c5\u03bd\u03bf\u03bb\u03b9\u03ba\u03ae\u03c2 peak GPU allocation.<\/p>\n<p>\u0397 \u03b4\u03b9\u03ac\u03ba\u03c1\u03b9\u03c3\u03b7 \u03b1\u03bd\u03ac\u03bc\u03b5\u03c3\u03b1 \u03c3\u03b5 cache footprint \u03ba\u03b1\u03b9 end-to-end \u03bc\u03bd\u03ae\u03bc\u03b7 \u03b5\u03af\u03bd\u03b1\u03b9 \u03ba\u03c1\u03af\u03c3\u03b9\u03bc\u03b7. \u038c\u03c0\u03c9\u03c2 \u03b4\u03b5\u03af\u03c7\u03bd\u03b5\u03b9 \u03ba\u03b1\u03b9 \u03c4\u03bf \u03b6\u03ae\u03c4\u03b7\u03bc\u03b1 \u03c4\u03bf\u03c5 <a href=\"https:\/\/twodots.gr\/moe-modela-stin-akri-cache-bandwidth-bottleneck\/\">cache \u03ba\u03b1\u03b9 bandwidth bottleneck \u03c3\u03b5 MoE inference<\/a>, \u03bc\u03b9\u03b1 \u03c4\u03bf\u03c0\u03b9\u03ba\u03ae \u03b2\u03b5\u03bb\u03c4\u03b9\u03c3\u03c4\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b7 \u03b4\u03b5\u03bd \u03b5\u03be\u03b1\u03c6\u03b1\u03bd\u03af\u03b6\u03b5\u03b9 \u03c4\u03b1 \u03ac\u03bb\u03bb\u03b1 \u03cc\u03c1\u03b9\u03b1 \u03c4\u03bf\u03c5 \u03c3\u03c5\u03c3\u03c4\u03ae\u03bc\u03b1\u03c4\u03bf\u03c2. \u03a4\u03bf memory budget \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03b4\u03bf\u03ba\u03b9\u03bc\u03b1\u03c3\u03c4\u03b5\u03af \u03bc\u03b5 \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc concurrency, \u03bc\u03ae\u03ba\u03b7 output \u03ba\u03b1\u03b9 fragmentation.<\/p>\n<h2 id=\"benchmark-setup\">\u03a4\u03b9 \u03b1\u03ba\u03c1\u03b9\u03b2\u03ce\u03c2 \u03bc\u03b5\u03c4\u03c1\u03ae\u03b8\u03b7\u03ba\u03b5 \u03c3\u03c4\u03bf benchmark<\/h2>\n<p>\u03a4\u03b1 \u03c0\u03b5\u03b9\u03c1\u03ac\u03bc\u03b1\u03c4\u03b1 \u03c4\u03b7\u03c2 \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1\u03c2 \u03ad\u03b3\u03b9\u03bd\u03b1\u03bd \u03bc\u03b5 Qwen\/Qwen2.5-3B \u03c3\u03b5 float16, \u03c3\u03b5 \u03bc\u03af\u03b1 NVIDIA GeForce RTX 4060 \u03bc\u03b5 8 GB VRAM. \u03a4\u03bf workload \u03c0\u03b5\u03c1\u03b9\u03bb\u03ac\u03bc\u03b2\u03b1\u03bd\u03b5 1.000 multiple-choice \u03b4\u03b5\u03af\u03b3\u03bc\u03b1\u03c4\u03b1 bug localization. \u039a\u03bf\u03b9\u03bd\u03cc code context \u03b5\u03c0\u03b1\u03bd\u03b1\u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03bf\u03cd\u03bd\u03c4\u03b1\u03bd \u03c3\u03b5 \u03bf\u03bc\u03ac\u03b4\u03b5\u03c2 \u03b5\u03c1\u03c9\u03c4\u03ae\u03c3\u03b5\u03c9\u03bd, \u03b4\u03b7\u03bc\u03b9\u03bf\u03c5\u03c1\u03b3\u03ce\u03bd\u03c4\u03b1\u03c2 cold request \u03b3\u03b9\u03b1 \u03c4\u03bf \u03c0\u03c1\u03ce\u03c4\u03bf \u03b4\u03b5\u03af\u03b3\u03bc\u03b1 \u03ba\u03b1\u03b9 warm-cache requests \u03b3\u03b9\u03b1 \u03cc\u03c3\u03b1 \u03b1\u03ba\u03bf\u03bb\u03bf\u03c5\u03b8\u03bf\u03cd\u03c3\u03b1\u03bd.<\/p>\n<p>\u03a4\u03b1 contexts \u03ba\u03c5\u03bc\u03b1\u03af\u03bd\u03bf\u03bd\u03c4\u03b1\u03bd \u03b1\u03c0\u03cc 163 \u03ad\u03c9\u03c2 \u03c0\u03ac\u03bd\u03c9 \u03b1\u03c0\u03cc 3.400 tokens. \u039f\u03b9 \u03c4\u03ad\u03c3\u03c3\u03b5\u03c1\u03b9\u03c2 \u03bf\u03bc\u03ac\u03b4\u03b5\u03c2 \u03bc\u03ae\u03ba\u03bf\u03c5\u03c2 \u03b5\u03af\u03c7\u03b1\u03bd 218 \u03b4\u03b5\u03af\u03b3\u03bc\u03b1\u03c4\u03b1 \u03ad\u03c9\u03c2 500 tokens, 210 \u03b1\u03c0\u03cc 500 \u03ad\u03c9\u03c2 1K, 204 \u03b1\u03c0\u03cc 1K \u03ad\u03c9\u03c2 2K \u03ba\u03b1\u03b9 368 \u03c0\u03ac\u03bd\u03c9 \u03b1\u03c0\u03cc 2K. \u03a3\u03c5\u03b3\u03ba\u03c1\u03af\u03b8\u03b7\u03ba\u03b1\u03bd full recomputation, vLLM prefix caching \u03ba\u03b1\u03b9 KVBoost \u03bc\u03b5 chunk size 128 \u03ba\u03b1\u03b9 CacheBlendRecompute.<\/p>\n<p>\u039f\u03b9 \u03b2\u03b1\u03c3\u03b9\u03ba\u03ad\u03c2 \u03bc\u03b5\u03c4\u03c1\u03b9\u03ba\u03ad\u03c2 \u03ae\u03c4\u03b1\u03bd exact-match accuracy, time-to-first-token, peak GPU memory \u03ba\u03b1\u03b9 \u03c0\u03bf\u03c3\u03bf\u03c3\u03c4\u03cc cache reuse. \u0397 \u03b5\u03c0\u03b9\u03bb\u03bf\u03b3\u03ae \u03b5\u03af\u03bd\u03b1\u03b9 \u03ba\u03b1\u03c4\u03ac\u03bb\u03bb\u03b7\u03bb\u03b7 \u03b3\u03b9\u03b1 \u03c4\u03bf \u03c3\u03c5\u03b3\u03ba\u03b5\u03ba\u03c1\u03b9\u03bc\u03ad\u03bd\u03bf \u03c3\u03cd\u03bd\u03c4\u03bf\u03bc\u03bf multiple-choice task, \u03b1\u03bb\u03bb\u03ac \u03b4\u03b5\u03bd \u03ba\u03b1\u03bb\u03cd\u03c0\u03c4\u03b5\u03b9 throughput \u03c5\u03c0\u03cc \u03c0\u03bf\u03bb\u03bb\u03bf\u03cd\u03c2 \u03c4\u03b1\u03c5\u03c4\u03cc\u03c7\u03c1\u03bf\u03bd\u03bf\u03c5\u03c2 \u03c7\u03c1\u03ae\u03c3\u03c4\u03b5\u03c2, p99 \u03c3\u03b5 distributed serving \u03ae \u03c0\u03bf\u03b9\u03cc\u03c4\u03b7\u03c4\u03b1 \u03bc\u03b5\u03b3\u03ac\u03bb\u03c9\u03bd \u03b5\u03bb\u03b5\u03cd\u03b8\u03b5\u03c1\u03c9\u03bd \u03b1\u03c0\u03b1\u03bd\u03c4\u03ae\u03c3\u03b5\u03c9\u03bd.<\/p>\n<p>\u0391\u03c5\u03c4\u03cc\u03c2 \u03b5\u03af\u03bd\u03b1\u03b9 \u03bf \u03bb\u03cc\u03b3\u03bf\u03c2 \u03c0\u03bf\u03c5 \u03b7 \u03b1\u03be\u03b9\u03bf\u03bb\u03cc\u03b3\u03b7\u03c3\u03b7 \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03b4\u03b9\u03b1\u03b2\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03bc\u03b1\u03b6\u03af \u03bc\u03b5 \u03c4\u03bf setup. \u038c\u03c0\u03c9\u03c2 \u03c3\u03c5\u03bc\u03b2\u03b1\u03af\u03bd\u03b5\u03b9 \u03cc\u03c4\u03b1\u03bd <a href=\"https:\/\/twodots.gr\/ai-benchmark-harness-allazei-nikiti\/\">\u03c4\u03bf benchmark harness \u03b1\u03bb\u03bb\u03ac\u03b6\u03b5\u03b9 \u03c4\u03bf\u03bd \u03bd\u03b9\u03ba\u03b7\u03c4\u03ae \u03b5\u03bd\u03cc\u03c2 AI \u03c3\u03c5\u03c3\u03c4\u03ae\u03bc\u03b1\u03c4\u03bf\u03c2<\/a>, \u03bf \u03b1\u03c1\u03b9\u03b8\u03bc\u03cc\u03c2 \u03ad\u03c7\u03b5\u03b9 \u03bd\u03cc\u03b7\u03bc\u03b1 \u03bc\u03cc\u03bd\u03bf \u03b1\u03bd workload, baseline, hardware \u03ba\u03b1\u03b9 output \u03bc\u03bf\u03c1\u03c6\u03ae \u03bc\u03bf\u03b9\u03ac\u03b6\u03bf\u03c5\u03bd \u03bc\u03b5 \u03c4\u03b7\u03bd \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03ae \u03c7\u03c1\u03ae\u03c3\u03b7.<\/p>\n<h2 id=\"apotelesmata-ttft\">\u03a4\u03b9 \u03b4\u03b5\u03af\u03c7\u03bd\u03bf\u03c5\u03bd \u03c4\u03b1 \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03ad\u03c3\u03bc\u03b1\u03c4\u03b1 latency \u03ba\u03b1\u03b9 \u03c0\u03bf\u03b9\u03cc\u03c4\u03b7\u03c4\u03b1\u03c2<\/h2>\n<p>\u039f \u03bc\u03ad\u03c3\u03bf\u03c2 TTFT \u03ae\u03c4\u03b1\u03bd 639,1 ms \u03b3\u03b9\u03b1 full recomputation, 165,5 ms \u03b3\u03b9\u03b1 vLLM prefix caching \u03ba\u03b1\u03b9 142,4 ms \u03b3\u03b9\u03b1 KVBoost. \u0397 \u03b4\u03b9\u03ac\u03bc\u03b5\u03c3\u03bf\u03c2 \u03ae\u03c4\u03b1\u03bd 440,3, 80,0 \u03ba\u03b1\u03b9 76,1 ms \u03b1\u03bd\u03c4\u03af\u03c3\u03c4\u03bf\u03b9\u03c7\u03b1, \u03b5\u03bd\u03ce \u03c4\u03bf p95 \u03ae\u03c4\u03b1\u03bd 1.702,0, 651,4 \u03ba\u03b1\u03b9 502,6 ms. \u039c\u03b5 \u03b2\u03ac\u03c3\u03b7 \u03b1\u03c5\u03c4\u03ad\u03c2 \u03c4\u03b9\u03c2 \u03c4\u03b9\u03bc\u03ad\u03c2, \u03c4\u03bf KVBoost \u03ad\u03b4\u03c9\u03c3\u03b5 4,49\u00d7 mean speedup \u03ad\u03bd\u03b1\u03bd\u03c4\u03b9 \u03c4\u03bf\u03c5 baseline \u03ba\u03b1\u03b9 16% \u03c7\u03b1\u03bc\u03b7\u03bb\u03cc\u03c4\u03b5\u03c1\u03bf mean TTFT \u03b1\u03c0\u03cc \u03c4\u03bf prefix cache.<\/p>\n<div class=\"td-chart td-chart--metrics\">\n<div class=\"td-chart-head\">\n<p class=\"td-chart-kicker\">KVBoost \u03c3\u03c4\u03bf \u03c3\u03c5\u03b3\u03ba\u03b5\u03ba\u03c1\u03b9\u03bc\u03ad\u03bd\u03bf \u03c0\u03b5\u03af\u03c1\u03b1\u03bc\u03b1<\/p>\n<p class=\"td-chart-title\">Latency \u03ba\u03b1\u03b9 \u03c0\u03bf\u03b9\u03cc\u03c4\u03b7\u03c4\u03b1 \u03c3\u03b5 1.000 \u03b4\u03b5\u03af\u03b3\u03bc\u03b1\u03c4\u03b1 bug localization<\/p>\n<p class=\"td-chart-intro\">\u039f\u03b9 \u03c4\u03b9\u03bc\u03ad\u03c2 \u03b1\u03c6\u03bf\u03c1\u03bf\u03cd\u03bd Qwen2.5-3B, float16 \u03ba\u03b1\u03b9 \u03bc\u03af\u03b1 RTX 4060. \u0394\u03b5\u03bd \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03bf\u03cd\u03bd \u03b3\u03b5\u03bd\u03b9\u03ba\u03ae \u03c0\u03c1\u03cc\u03b2\u03bb\u03b5\u03c8\u03b7 \u03b3\u03b9\u03b1 \u03ac\u03bb\u03bb\u03bf \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03bf, hardware \u03ae workload.<\/p>\n<\/div>\n<div class=\"td-chart-body\">\n<div class=\"td-metric-grid\">\n<div class=\"td-metric-card\"><span class=\"td-metric-value\">142,4 ms<\/span><span class=\"td-metric-label\">\u039c\u03ad\u03c3\u03bf\u03c2 TTFT \u03bc\u03b5 KVBoost \u03ad\u03bd\u03b1\u03bd\u03c4\u03b9 639,1 ms \u03b3\u03b9\u03b1 full recomputation<\/span><\/div>\n<div class=\"td-metric-card\"><span class=\"td-metric-value\">4,49\u00d7<\/span><span class=\"td-metric-label\">Mean TTFT speedup \u03c4\u03bf\u03c5 KVBoost \u03b1\u03c0\u03ad\u03bd\u03b1\u03bd\u03c4\u03b9 \u03c3\u03c4\u03bf\u03bd \u03c0\u03bb\u03ae\u03c1\u03b7 \u03b5\u03c0\u03b1\u03bd\u03c5\u03c0\u03bf\u03bb\u03bf\u03b3\u03b9\u03c3\u03bc\u03cc<\/span><\/div>\n<div class=\"td-metric-card\"><span class=\"td-metric-value\">16%<\/span><span class=\"td-metric-label\">\u03a7\u03b1\u03bc\u03b7\u03bb\u03cc\u03c4\u03b5\u03c1\u03bf\u03c2 \u03bc\u03ad\u03c3\u03bf\u03c2 TTFT \u03b1\u03c0\u03cc \u03c4\u03bf vLLM prefix caching \u03c3\u03c4\u03bf \u03af\u03b4\u03b9\u03bf benchmark<\/span><\/div>\n<div class=\"td-metric-card\"><span class=\"td-metric-value\">99,2%<\/span><span class=\"td-metric-label\">Exact-match accuracy \u03ad\u03bd\u03b1\u03bd\u03c4\u03b9 99,1% \u03b3\u03b9\u03b1 full recomputation \u03ba\u03b1\u03b9 vLLM<\/span><\/div>\n<\/div>\n<\/div>\n<\/div>\n<p>\u0397 \u03bc\u03b9\u03ba\u03c1\u03ae \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03ac accuracy \u03b4\u03b5\u03bd \u03c0\u03b1\u03c1\u03bf\u03c5\u03c3\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc \u03c4\u03bf\u03bd \u03c3\u03c5\u03b3\u03b3\u03c1\u03b1\u03c6\u03ad\u03b1 \u03c9\u03c2 \u03c3\u03c5\u03c3\u03c4\u03b7\u03bc\u03b1\u03c4\u03b9\u03ba\u03ae \u03b2\u03b5\u03bb\u03c4\u03af\u03c9\u03c3\u03b7 \u03c0\u03bf\u03b9\u03cc\u03c4\u03b7\u03c4\u03b1\u03c2. \u03a4\u03bf \u03c3\u03c9\u03c3\u03c4\u03cc \u03c3\u03c5\u03bc\u03c0\u03ad\u03c1\u03b1\u03c3\u03bc\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 \u03cc\u03c4\u03b9 \u03b4\u03b5\u03bd \u03b1\u03bd\u03b9\u03c7\u03bd\u03b5\u03cd\u03c4\u03b7\u03ba\u03b5 regression \u03c3\u03c4\u03bf \u03b1\u03be\u03b9\u03bf\u03bb\u03bf\u03b3\u03b7\u03bc\u03ad\u03bd\u03bf task. \u03a4\u03bf average reuse \u03ae\u03c4\u03b1\u03bd 36,4% \u03b3\u03b9\u03b1 KVBoost \u03ba\u03b1\u03b9 39,5% \u03b3\u03b9\u03b1 vLLM, \u03c0\u03b1\u03c1\u03cc\u03c4\u03b9 \u03c4\u03bf KVBoost \u03b5\u03af\u03c7\u03b5 \u03c7\u03b1\u03bc\u03b7\u03bb\u03cc\u03c4\u03b5\u03c1\u03bf TTFT \u03c3\u03c4\u03b7 \u03c3\u03c5\u03b3\u03ba\u03b5\u03ba\u03c1\u03b9\u03bc\u03ad\u03bd\u03b7 \u03c5\u03bb\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b7.<\/p>\n<p>\u0397 peak GPU memory \u03bc\u03b5\u03b9\u03ce\u03b8\u03b7\u03ba\u03b5 \u03bc\u03cc\u03bd\u03bf \u03ba\u03b1\u03c4\u03ac 14,8 MB \u03ae 0,24%. \u03a4\u03bf \u03ba\u03cd\u03c1\u03b9\u03bf \u03bc\u03b5\u03c4\u03c1\u03b7\u03bc\u03ad\u03bd\u03bf \u03cc\u03c6\u03b5\u03bb\u03bf\u03c2 \u03ae\u03c4\u03b1\u03bd \u03b7 \u03b1\u03c0\u03bf\u03c6\u03c5\u03b3\u03ae recomputation \u03ba\u03b1\u03b9 \u03cc\u03c7\u03b9 \u03bc\u03b9\u03b1 \u03b8\u03b5\u03b1\u03bc\u03b1\u03c4\u03b9\u03ba\u03ae \u03c0\u03c4\u03ce\u03c3\u03b7 \u03c4\u03b7\u03c2 \u03c3\u03c5\u03bd\u03bf\u03bb\u03b9\u03ba\u03ae\u03c2 \u03bc\u03bd\u03ae\u03bc\u03b7\u03c2. \u0397 \u03c0\u03bb\u03b7\u03c1\u03bf\u03c6\u03bf\u03c1\u03af\u03b1 \u03b1\u03c5\u03c4\u03ae \u03b1\u03c0\u03bf\u03c4\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bc\u03b9\u03b1 \u03c3\u03c5\u03bd\u03b7\u03b8\u03b9\u03c3\u03bc\u03ad\u03bd\u03b7 \u03c5\u03c0\u03b5\u03c1\u03b5\u03c1\u03bc\u03b7\u03bd\u03b5\u03af\u03b1: quantized cached chunks \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03c7\u03c9\u03c1\u03bf\u03cd\u03bd \u03c0\u03b5\u03c1\u03b9\u03c3\u03c3\u03cc\u03c4\u03b5\u03c1\u03b1, \u03c7\u03c9\u03c1\u03af\u03c2 \u03b7 \u03c3\u03c5\u03bd\u03bf\u03bb\u03b9\u03ba\u03ae \u03b5\u03c6\u03b1\u03c1\u03bc\u03bf\u03b3\u03ae \u03bd\u03b1 \u03ba\u03b1\u03c4\u03b1\u03bd\u03b1\u03bb\u03ce\u03bd\u03b5\u03b9 \u03b1\u03bd\u03b1\u03bb\u03bf\u03b3\u03b9\u03ba\u03ac \u03bb\u03b9\u03b3\u03cc\u03c4\u03b5\u03c1\u03b7 VRAM.<\/p>\n<aside class=\"td-article-note\"><strong>\u038c\u03c1\u03b9\u03bf \u03b5\u03c1\u03bc\u03b7\u03bd\u03b5\u03af\u03b1\u03c2:<\/strong> \u03c4\u03b1 142,4 ms, \u03c4\u03bf 4,49\u00d7 \u03ba\u03b1\u03b9 \u03c4\u03bf 99,2% \u03b5\u03af\u03bd\u03b1\u03b9 \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03ad\u03c3\u03bc\u03b1\u03c4\u03b1 \u03b5\u03bd\u03cc\u03c2 single-GPU, single-model, multiple-choice workload. \u0394\u03b5\u03bd \u03bc\u03b5\u03c4\u03b1\u03c6\u03ad\u03c1\u03bf\u03bd\u03c4\u03b1\u03b9 \u03b1\u03c5\u03c4\u03cc\u03bc\u03b1\u03c4\u03b1 \u03c3\u03b5 long-form assistants, tensor-parallel serving \u03ae prompts \u03ac\u03bd\u03c9 \u03c4\u03c9\u03bd 8K tokens.<\/aside>\n<h2 id=\"pote-kerdizei\">\u03a0\u03cc\u03c4\u03b5 \u03ba\u03b5\u03c1\u03b4\u03af\u03b6\u03b5\u03b9 \u2014 \u03ba\u03b1\u03b9 \u03c0\u03cc\u03c4\u03b5 \u03cc\u03c7\u03b9<\/h2>\n<p>\u03a3\u03c4\u03bf bucket \u03ad\u03c9\u03c2 500 tokens, \u03c4\u03bf KVBoost \u03b5\u03af\u03c7\u03b5 \u03bc\u03ad\u03c3\u03bf TTFT 48,8 ms \u03ad\u03bd\u03b1\u03bd\u03c4\u03b9 74,9 ms \u03b3\u03b9\u03b1 vLLM, \u03c0\u03bb\u03b5\u03bf\u03bd\u03ad\u03ba\u03c4\u03b7\u03bc\u03b1 1,53\u00d7. \u03a3\u03c4\u03b1 500\u20131K \u03ae\u03c4\u03b1\u03bd 73,7 \u03ad\u03bd\u03b1\u03bd\u03c4\u03b9 100,5 ms, \u03c3\u03c4\u03b1 1K\u20132K 118,9 \u03ad\u03bd\u03b1\u03bd\u03c4\u03b9 138,8 ms \u03ba\u03b1\u03b9 \u03c0\u03ac\u03bd\u03c9 \u03b1\u03c0\u03cc 2K 250,2 \u03ad\u03bd\u03b1\u03bd\u03c4\u03b9 271,2 ms. \u03a4\u03bf \u03c0\u03bb\u03b5\u03bf\u03bd\u03ad\u03ba\u03c4\u03b7\u03bc\u03b1 \u03ad\u03bd\u03b1\u03bd\u03c4\u03b9 prefix caching \u03bc\u03b5\u03b9\u03ce\u03b8\u03b7\u03ba\u03b5 \u03cc\u03c3\u03bf \u03c4\u03bf \u03ba\u03bf\u03b9\u03bd\u03cc leading prefix \u03ba\u03ac\u03bb\u03c5\u03c0\u03c4\u03b5 \u03bc\u03b5\u03b3\u03b1\u03bb\u03cd\u03c4\u03b5\u03c1\u03bf \u03bc\u03ad\u03c1\u03bf\u03c2 \u03c4\u03b7\u03c2 \u03bc\u03b1\u03ba\u03c1\u03ac\u03c2 \u03b5\u03b9\u03c3\u03cc\u03b4\u03bf\u03c5.<\/p>\n<p>\u0391\u03c0\u03ad\u03bd\u03b1\u03bd\u03c4\u03b9 \u03c3\u03c4\u03bf\u03bd \u03c0\u03bb\u03ae\u03c1\u03b7 \u03b5\u03c0\u03b1\u03bd\u03c5\u03c0\u03bf\u03bb\u03bf\u03b3\u03b9\u03c3\u03bc\u03cc, \u03c4\u03bf speedup \u03b1\u03c5\u03be\u03ae\u03b8\u03b7\u03ba\u03b5 \u03b1\u03c0\u03cc 3,34\u00d7 \u03c3\u03c4\u03bf \u03bc\u03b9\u03ba\u03c1\u03cc\u03c4\u03b5\u03c1\u03bf bucket \u03ad\u03c9\u03c2 4,84\u00d7 \u03c3\u03c4\u03bf 2K+. \u0397 \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1 \u03b5\u03bd\u03c4\u03bf\u03c0\u03af\u03b6\u03b5\u03b9 \u03c4\u03bf sweet spot \u03c3\u03b5 workloads \u03cc\u03c0\u03bf\u03c5 \u03c0\u03bf\u03bb\u03bb\u03ac \u03ba\u03bf\u03b9\u03bd\u03ac segments \u03b5\u03bc\u03c6\u03b1\u03bd\u03af\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 \u03c3\u03b5 \u03bc\u03b7 \u03b1\u03c1\u03c7\u03b9\u03ba\u03ad\u03c2 \u03b8\u03ad\u03c3\u03b5\u03b9\u03c2, \u03c4\u03b1 system prompts \u03b1\u03ba\u03bf\u03bb\u03bf\u03c5\u03b8\u03bf\u03cd\u03bd \u03bc\u03b5\u03c4\u03b1\u03b2\u03b1\u03bb\u03bb\u03cc\u03bc\u03b5\u03bd\u03bf preamble \u03ae \u03b3\u03af\u03bd\u03b5\u03c4\u03b1\u03b9 batch generation \u03c0\u03ac\u03bd\u03c9 \u03c3\u03b5 \u03c3\u03c4\u03b1\u03b8\u03b5\u03c1\u03cc corpus.<\/p>\n<p>\u03a4\u03bf \u03b1\u03c0\u03bb\u03cc prefix cache \u03c0\u03b1\u03c1\u03b1\u03bc\u03ad\u03bd\u03b5\u03b9 \u03c0\u03c1\u03bf\u03c4\u03b9\u03bc\u03cc\u03c4\u03b5\u03c1\u03bf \u03cc\u03c4\u03b1\u03bd \u03c5\u03c0\u03ac\u03c1\u03c7\u03b5\u03b9 \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03ac \u03ba\u03bf\u03b9\u03bd\u03ae \u03b1\u03c1\u03c7\u03ae \u03ba\u03b1\u03b9 \u03b1\u03c0\u03b1\u03b9\u03c4\u03b5\u03af\u03c4\u03b1\u03b9 \u03b1\u03ba\u03c1\u03b9\u03b2\u03ae\u03c2 positional correctness. \u0394\u03b5\u03bd \u03c0\u03bb\u03b7\u03c1\u03ce\u03bd\u03b5\u03b9 probe \u03bf\u03cd\u03c4\u03b5 seam-repair overhead. \u0391\u03bd \u03c4\u03b1 outputs \u03b5\u03af\u03bd\u03b1\u03b9 \u03c0\u03bf\u03bb\u03cd \u03bc\u03b5\u03b3\u03ac\u03bb\u03b1 \u03ba\u03b1\u03b9 \u03c4\u03bf decode \u03ba\u03c5\u03c1\u03b9\u03b1\u03c1\u03c7\u03b5\u03af \u03c3\u03c4\u03bf \u03c3\u03c5\u03bd\u03bf\u03bb\u03b9\u03ba\u03cc latency, \u03ba\u03b1\u03bc\u03af\u03b1 \u03b1\u03c0\u03cc \u03c4\u03b9\u03c2 \u03b4\u03cd\u03bf prefill optimizations \u03b4\u03b5\u03bd \u03b1\u03c1\u03ba\u03b5\u03af \u03b1\u03c0\u03cc \u03bc\u03cc\u03bd\u03b7 \u03c4\u03b7\u03c2.<\/p>\n<p>\u0397 \u03b1\u03c0\u03cc\u03c6\u03b1\u03c3\u03b7 \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03b2\u03b1\u03c3\u03b9\u03c3\u03c4\u03b5\u03af \u03c3\u03b5 prompt traces. \u03a7\u03c9\u03c1\u03af\u03c2 \u03bd\u03b1 \u03bc\u03b5\u03c4\u03c1\u03b7\u03b8\u03b5\u03af \u03c0\u03bf\u03b9\u03bf \u03c0\u03b5\u03c1\u03b9\u03b5\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf \u03b5\u03c0\u03b1\u03bd\u03b1\u03bb\u03b1\u03bc\u03b2\u03ac\u03bd\u03b5\u03c4\u03b1\u03b9, \u03c3\u03b5 \u03c0\u03bf\u03b9\u03b5\u03c2 \u03b8\u03ad\u03c3\u03b5\u03b9\u03c2, \u03c0\u03cc\u03c3\u03bf \u03c3\u03c5\u03c7\u03bd\u03ac \u03c4\u03bf traffic \u03b5\u03af\u03bd\u03b1\u03b9 warm \u03ba\u03b1\u03b9 \u03c4\u03b9 \u03c0\u03bf\u03c3\u03bf\u03c3\u03c4\u03cc \u03c4\u03bf\u03c5 \u03c7\u03c1\u03cc\u03bd\u03bf\u03c5 \u03b1\u03bd\u03ae\u03ba\u03b5\u03b9 \u03c3\u03c4\u03bf prefill, \u03b7 \u03b5\u03c0\u03b9\u03bb\u03bf\u03b3\u03ae cache \u03bc\u03ad\u03bd\u03b5\u03b9 \u03c5\u03c0\u03cc\u03b8\u03b5\u03c3\u03b7.<\/p>\n<h2 id=\"symvatotita-periorismoi\">\u03a3\u03c5\u03bc\u03b2\u03b1\u03c4\u03cc\u03c4\u03b7\u03c4\u03b1 \u03ba\u03b1\u03b9 \u03c4\u03b5\u03c7\u03bd\u03b9\u03ba\u03bf\u03af \u03c0\u03b5\u03c1\u03b9\u03bf\u03c1\u03b9\u03c3\u03bc\u03bf\u03af<\/h2>\n<p>\u0397 \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1 \u03c3\u03c4\u03bf\u03c7\u03b5\u03cd\u03b5\u03b9 Hugging Face decoder models \u03bc\u03b5 RoPE \u03ba\u03b1\u03b9 \u03b4\u03b9\u03b5\u03c0\u03b1\u03c6\u03ae past_key_values. \u0391\u03bd\u03b1\u03c6\u03ad\u03c1\u03b5\u03b9 \u03bf\u03b9\u03ba\u03bf\u03b3\u03ad\u03bd\u03b5\u03b9\u03b5\u03c2 Qwen2, LLaMA, Mistral, Mixtral, Gemma, Phi, StableLM \u03ba\u03b1\u03b9 InternLM. \u0394\u03b5\u03bd \u03ba\u03b1\u03bb\u03cd\u03c0\u03c4\u03b5\u03b9 MPT \u03ae Falcon \u03bc\u03b5 ALiBi, GPT-2 \u03bc\u03b5 learned absolute positions, \u03bf\u03cd\u03c4\u03b5 Mistral \u03cc\u03c4\u03b1\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03b5\u03bd\u03b5\u03c1\u03b3\u03cc sliding-window attention \u03c3\u03c4\u03b7\u03bd \u03c0\u03b1\u03c1\u03bf\u03c5\u03c3\u03b9\u03b1\u03b6\u03cc\u03bc\u03b5\u03bd\u03b7 \u03c5\u03bb\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b7.<\/p>\n<p>\u0397 \u03c4\u03b5\u03c7\u03bd\u03b9\u03ba\u03ae \u03b5\u03af\u03bd\u03b1\u03b9 \u03bb\u03bf\u03b9\u03c0\u03cc\u03bd RoPE-only \u03ba\u03b1\u03b9 model compatibility \u03b4\u03b5\u03bd \u03c3\u03b7\u03bc\u03b1\u03af\u03bd\u03b5\u03b9 validated quality. \u03a4\u03bf chunk size \u03c0\u03b1\u03c1\u03b1\u03bc\u03ad\u03bd\u03b5\u03b9 \u03b5\u03c5\u03b1\u03af\u03c3\u03b8\u03b7\u03c4\u03b7 \u03c0\u03b1\u03c1\u03ac\u03bc\u03b5\u03c4\u03c1\u03bf\u03c2: \u03c0\u03bf\u03bb\u03cd \u03bc\u03b9\u03ba\u03c1\u03ac chunks \u03b1\u03c5\u03be\u03ac\u03bd\u03bf\u03c5\u03bd \u03c4\u03b1 seams \u03ba\u03b1\u03b9 \u03c4\u03bf repair cost, \u03b5\u03bd\u03ce \u03c0\u03bf\u03bb\u03cd \u03bc\u03b5\u03b3\u03ac\u03bb\u03b1 \u03bc\u03b5\u03b9\u03ce\u03bd\u03bf\u03c5\u03bd \u03c4\u03b7\u03bd \u03c0\u03b9\u03b8\u03b1\u03bd\u03cc\u03c4\u03b7\u03c4\u03b1 hit. \u039f \u03c3\u03c9\u03c3\u03c4\u03cc\u03c2 \u03c3\u03c5\u03bc\u03b2\u03b9\u03b2\u03b1\u03c3\u03bc\u03cc\u03c2 \u03b5\u03be\u03b1\u03c1\u03c4\u03ac\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc \u03c4\u03b7 \u03b4\u03bf\u03bc\u03ae \u03c4\u03c9\u03bd \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03ce\u03bd prompts.<\/p>\n<p>\u03a4\u03bf \u03c0\u03b5\u03af\u03c1\u03b1\u03bc\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 single-GPU \u03ba\u03b1\u03b9 \u03b7 \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1 \u03b4\u03b5\u03bd \u03c0\u03b1\u03c1\u03bf\u03c5\u03c3\u03b9\u03ac\u03b6\u03b5\u03b9 \u03ba\u03b1\u03c4\u03b1\u03bd\u03bf\u03bc\u03ae cache shards \u03b3\u03b9\u03b1 tensor parallelism. \u0395\u03c0\u03af\u03c3\u03b7\u03c2 \u03b1\u03bd\u03b1\u03c6\u03ad\u03c1\u03b5\u03b9 \u03cc\u03c4\u03b9 \u03c4\u03bf probe cost \u03bc\u03b5\u03b3\u03b1\u03bb\u03ce\u03bd\u03b5\u03b9 \u03c3\u03b5 contexts \u03c0\u03ac\u03bd\u03c9 \u03b1\u03c0\u03cc 8K tokens. \u0397 \u03b1\u03be\u03b9\u03bf\u03bb\u03cc\u03b3\u03b7\u03c3\u03b7 \u03c0\u03b5\u03c1\u03b9\u03bf\u03c1\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 \u03c3\u03b5 bug localization \u03bc\u03b5 \u03c3\u03cd\u03bd\u03c4\u03bf\u03bc\u03b1 outputs \u03ba\u03b1\u03b9 \u03b4\u03b5\u03bd \u03b1\u03c0\u03bf\u03b4\u03b5\u03b9\u03ba\u03bd\u03cd\u03b5\u03b9 \u03af\u03b4\u03b9\u03b1 \u03c3\u03c5\u03bc\u03c0\u03b5\u03c1\u03b9\u03c6\u03bf\u03c1\u03ac \u03c3\u03b5 code completion, multi-document summarization \u03ae \u03bc\u03b1\u03ba\u03c1\u03bf\u03c3\u03ba\u03b5\u03bb\u03ae generation.<\/p>\n<p>\u03a4\u03bf \u03b4\u03b7\u03bc\u03cc\u03c3\u03b9\u03bf repository \u03ad\u03c7\u03b5\u03b9 \u03b5\u03be\u03b5\u03bb\u03b9\u03c7\u03b8\u03b5\u03af \u03ba\u03b1\u03b9 \u03c0\u03b5\u03c1\u03b9\u03b3\u03c1\u03ac\u03c6\u03b5\u03b9 \u03b5\u03c0\u03b9\u03c0\u03bb\u03ad\u03bf\u03bd \u03b4\u03c5\u03bd\u03b1\u03c4\u03cc\u03c4\u03b7\u03c4\u03b5\u03c2, \u03cc\u03c0\u03c9\u03c2 server, batching \u03ba\u03b1\u03b9 attention optimizations. \u0393\u03b9\u03b1 \u03c0\u03b9\u03c3\u03c4\u03ae \u03b1\u03bd\u03b1\u03c0\u03b1\u03c1\u03b1\u03b3\u03c9\u03b3\u03ae \u03c4\u03c9\u03bd \u03b1\u03c1\u03b9\u03b8\u03bc\u03ce\u03bd \u03c4\u03bf\u03c5 paper, \u03bc\u03b9\u03b1 \u03bf\u03bc\u03ac\u03b4\u03b1 \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03ba\u03bb\u03b5\u03b9\u03b4\u03ce\u03c3\u03b5\u03b9 version, commit, dependencies \u03ba\u03b1\u03b9 benchmark configuration \u03b1\u03bd\u03c4\u03af \u03bd\u03b1 \u03c5\u03c0\u03bf\u03b8\u03ad\u03c3\u03b5\u03b9 \u03cc\u03c4\u03b9 \u03c4\u03bf \u03c3\u03b7\u03bc\u03b5\u03c1\u03b9\u03bd\u03cc main branch \u03b5\u03af\u03bd\u03b1\u03b9 \u03c4\u03b1\u03c5\u03c4\u03cc\u03c3\u03b7\u03bc\u03bf \u03bc\u03b5 \u03c4\u03bf \u03c0\u03b5\u03b9\u03c1\u03b1\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc artifact.<\/p>\n<h2 id=\"rag-agents-proionta\">\u03a4\u03b9 \u03c3\u03b7\u03bc\u03b1\u03af\u03bd\u03b5\u03b9 \u03b3\u03b9\u03b1 RAG, agents \u03ba\u03b1\u03b9 AI \u03c0\u03c1\u03bf\u03ca\u03cc\u03bd\u03c4\u03b1<\/h2>\n<p>\u03a3\u03c4\u03bf RAG, \u03c4\u03bf \u03af\u03b4\u03b9\u03bf retrieved document \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b5\u03bc\u03c6\u03b1\u03bd\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 \u03c3\u03b5 \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03b5\u03c4\u03b9\u03ba\u03ae \u03c3\u03b5\u03b9\u03c1\u03ac \u03ae \u03bc\u03b5\u03c4\u03ac \u03b1\u03c0\u03cc \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03b5\u03c4\u03b9\u03ba\u03ad\u03c2 \u03bf\u03b4\u03b7\u03b3\u03af\u03b5\u03c2. \u0397 warm() \u03b4\u03b9\u03b5\u03c0\u03b1\u03c6\u03ae \u03ad\u03c7\u03b5\u03b9 \u03c3\u03c7\u03b5\u03b4\u03b9\u03b1\u03c3\u03c4\u03b5\u03af \u03b3\u03b9\u03b1 \u03bd\u03b1 \u03c0\u03c1\u03bf\u03c6\u03bf\u03c1\u03c4\u03ce\u03bd\u03b5\u03b9 \u03ba\u03bf\u03b9\u03bd\u03cc \u03c5\u03bb\u03b9\u03ba\u03cc, \u03ce\u03c3\u03c4\u03b5 \u03b5\u03c0\u03cc\u03bc\u03b5\u03bd\u03b1 prompts \u03bd\u03b1 \u03b1\u03bd\u03b1\u03ba\u03c4\u03bf\u03cd\u03bd \u03c4\u03b1 KV tensors \u03b1\u03bd\u03c4\u03af \u03bd\u03b1 \u03c4\u03b1 \u03c5\u03c0\u03bf\u03bb\u03bf\u03b3\u03af\u03b6\u03bf\u03c5\u03bd \u03be\u03b1\u03bd\u03ac. \u0391\u03c5\u03c4\u03cc \u03b5\u03af\u03bd\u03b1\u03b9 \u03b9\u03b4\u03b9\u03b1\u03af\u03c4\u03b5\u03c1\u03b1 \u03c3\u03c7\u03b5\u03c4\u03b9\u03ba\u03cc \u03cc\u03c4\u03b1\u03bd \u03ad\u03bd\u03b1 <a href=\"https:\/\/twodots.gr\/schemarouter-ai-agents-field-aware-rag\/\">field-aware RAG \u03c3\u03cd\u03c3\u03c4\u03b7\u03bc\u03b1 \u03c0\u03b5\u03c1\u03b9\u03bf\u03c1\u03af\u03b6\u03b5\u03b9 \u03c4\u03bf context \u03c3\u03c4\u03b1 \u03c3\u03c9\u03c3\u03c4\u03ac \u03b4\u03b5\u03b4\u03bf\u03bc\u03ad\u03bd\u03b1<\/a> \u03b1\u03bb\u03bb\u03ac \u03b4\u03b9\u03b1\u03c4\u03b7\u03c1\u03b5\u03af \u03b5\u03c0\u03b1\u03bd\u03b1\u03bb\u03b1\u03bc\u03b2\u03b1\u03bd\u03cc\u03bc\u03b5\u03bd\u03b1 document chunks.<\/p>\n<p>\u0393\u03b9\u03b1 AI agents, \u03c3\u03c4\u03b1\u03b8\u03b5\u03c1\u03ad\u03c2 \u03c0\u03bf\u03bb\u03b9\u03c4\u03b9\u03ba\u03ad\u03c2, tool descriptions \u03ba\u03b1\u03b9 \u03c4\u03bc\u03ae\u03bc\u03b1\u03c4\u03b1 \u03b9\u03c3\u03c4\u03bf\u03c1\u03b9\u03ba\u03bf\u03cd \u03bc\u03c0\u03bf\u03c1\u03bf\u03cd\u03bd \u03b8\u03b5\u03c9\u03c1\u03b7\u03c4\u03b9\u03ba\u03ac \u03bd\u03b1 \u03b4\u03ce\u03c3\u03bf\u03c5\u03bd content hits \u03b1\u03ba\u03cc\u03bc\u03b7 \u03ba\u03b9 \u03cc\u03c4\u03b1\u03bd \u03b4\u03b5\u03bd \u03b2\u03c1\u03af\u03c3\u03ba\u03bf\u03bd\u03c4\u03b1\u03b9 \u03c3\u03c4\u03b7\u03bd \u03b1\u03c1\u03c7\u03ae. \u0397 \u03af\u03b4\u03b9\u03b1 \u03b9\u03b4\u03ad\u03b1 \u03b5\u03bd\u03b4\u03b9\u03b1\u03c6\u03ad\u03c1\u03b5\u03b9 \u03ad\u03bd\u03b1 <a href=\"https:\/\/twodots.gr\/private-chatbot-rag-knowledge-base-business\/\">private chatbot \u03c0\u03bf\u03c5 \u03b1\u03c0\u03b1\u03bd\u03c4\u03ac \u03b1\u03c0\u03cc \u03b5\u03c4\u03b1\u03b9\u03c1\u03b9\u03ba\u03ae \u03b3\u03bd\u03ce\u03c3\u03b7<\/a>, \u03cc\u03c0\u03bf\u03c5 \u03c0\u03bf\u03bb\u03bb\u03bf\u03af \u03c7\u03c1\u03ae\u03c3\u03c4\u03b5\u03c2 \u03c1\u03c9\u03c4\u03bf\u03cd\u03bd \u03c0\u03ac\u03bd\u03c9 \u03c3\u03c4\u03b1 \u03af\u03b4\u03b9\u03b1 \u03ad\u03b3\u03b3\u03c1\u03b1\u03c6\u03b1 \u03bc\u03b5 \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03b5\u03c4\u03b9\u03ba\u03cc conversation context.<\/p>\n<p>\u0391\u03c5\u03c4\u03ad\u03c2 \u03b5\u03af\u03bd\u03b1\u03b9 \u03b5\u03c6\u03b1\u03c1\u03bc\u03bf\u03b3\u03ad\u03c2 \u03c4\u03b7\u03c2 \u03b1\u03c1\u03c7\u03b9\u03c4\u03b5\u03ba\u03c4\u03bf\u03bd\u03b9\u03ba\u03ae\u03c2, \u03cc\u03c7\u03b9 \u03b1\u03c0\u03cc\u03b4\u03b5\u03b9\u03be\u03b7 \u03cc\u03c4\u03b9 \u03ba\u03ac\u03b8\u03b5 e-commerce assistant \u03b8\u03b1 \u03b3\u03af\u03bd\u03b5\u03b9 4,49\u00d7 \u03c4\u03b1\u03c7\u03cd\u03c4\u03b5\u03c1\u03bf. \u03a4\u03bf \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc \u03cc\u03c6\u03b5\u03bb\u03bf\u03c2 \u03b5\u03be\u03b1\u03c1\u03c4\u03ac\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc repetition topology, tokenizer alignment, warm traffic, disk promotions, model size \u03ba\u03b1\u03b9 repair overhead. \u03a7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 replay \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03ce\u03bd traces \u03b1\u03bd\u03c4\u03af \u03b3\u03b9\u03b1 \u03c3\u03c5\u03bd\u03b8\u03b5\u03c4\u03b9\u03ba\u03ae \u03bc\u03b5\u03c4\u03b1\u03c6\u03bf\u03c1\u03ac \u03c4\u03bf\u03c5 benchmark.<\/p>\n<p>\u03a3\u03c4\u03bf \u03b5\u03c0\u03af\u03c0\u03b5\u03b4\u03bf \u03c0\u03c1\u03bf\u03ca\u03cc\u03bd\u03c4\u03bf\u03c2, \u03b7 \u03c7\u03b1\u03bc\u03b7\u03bb\u03cc\u03c4\u03b5\u03c1\u03b7 TTFT \u03ad\u03c7\u03b5\u03b9 \u03b1\u03be\u03af\u03b1 \u03cc\u03c4\u03b1\u03bd \u03bf \u03c7\u03c1\u03ae\u03c3\u03c4\u03b7\u03c2 \u03b1\u03bd\u03c4\u03b9\u03bb\u03b1\u03bc\u03b2\u03ac\u03bd\u03b5\u03c4\u03b1\u03b9 \u03c4\u03b7\u03bd \u03b1\u03bd\u03b1\u03bc\u03bf\u03bd\u03ae \u03c0\u03c1\u03b9\u03bd \u03b1\u03c1\u03c7\u03af\u03c3\u03b5\u03b9 \u03b7 \u03b1\u03c0\u03ac\u03bd\u03c4\u03b7\u03c3\u03b7. \u0391\u03bd \u03cc\u03bc\u03c9\u03c2 \u03b7 \u03b1\u03c0\u03cc\u03ba\u03c1\u03b9\u03c3\u03b7 \u03c0\u03b1\u03c1\u03ac\u03b3\u03b5\u03b9 \u03b5\u03ba\u03b1\u03c4\u03bf\u03bd\u03c4\u03ac\u03b4\u03b5\u03c2 tokens, \u03c4\u03bf decode throughput, \u03c4\u03bf batching \u03ba\u03b1\u03b9 \u03b7 \u03bf\u03c5\u03c1\u03ac \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03ad\u03c7\u03bf\u03c5\u03bd \u03bc\u03b5\u03b3\u03b1\u03bb\u03cd\u03c4\u03b5\u03c1\u03b7 \u03b5\u03c0\u03af\u03b4\u03c1\u03b1\u03c3\u03b7 \u03c3\u03c4\u03b7 \u03c3\u03c5\u03bd\u03bf\u03bb\u03b9\u03ba\u03ae \u03b5\u03bc\u03c0\u03b5\u03b9\u03c1\u03af\u03b1.<\/p>\n<h2 id=\"production-risk\">\u0391\u03c0\u03cc \u03c4\u03bf benchmark \u03c3\u03b5 \u03b5\u03bb\u03b5\u03b3\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf production rollout<\/h2>\n<p>\u03a4\u03bf \u03b1\u03c3\u03c6\u03b1\u03bb\u03ad\u03c2 rollout \u03be\u03b5\u03ba\u03b9\u03bd\u03ac \u03bc\u03b5 trace analysis \u03ba\u03b1\u03b9 exact-only mode. \u0397 \u03bf\u03bc\u03ac\u03b4\u03b1 \u03c7\u03b1\u03c1\u03c4\u03bf\u03b3\u03c1\u03b1\u03c6\u03b5\u03af \u03ba\u03bf\u03b9\u03bd\u03ac leading prefixes, chunks \u03c0\u03bf\u03c5 \u03bc\u03b5\u03c4\u03b1\u03ba\u03b9\u03bd\u03bf\u03cd\u03bd\u03c4\u03b1\u03b9 \u03ba\u03b1\u03b9 \u03c0\u03b5\u03c1\u03b9\u03b5\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf \u03c0\u03bf\u03c5 \u03b1\u03bb\u03bb\u03ac\u03b6\u03b5\u03b9 \u03ba\u03b1\u03c4\u03ac \u03bb\u03af\u03b3\u03b1 tokens. \u03a3\u03c4\u03b7 \u03c3\u03c5\u03bd\u03ad\u03c7\u03b5\u03b9\u03b1 \u03c3\u03c5\u03b3\u03ba\u03c1\u03af\u03bd\u03b5\u03b9 full recomputation, \u03c5\u03c0\u03ac\u03c1\u03c7\u03bf\u03bd prefix cache \u03ba\u03b1\u03b9 KVBoost \u03c3\u03c4\u03bf \u03af\u03b4\u03b9\u03bf hardware, \u03bc\u03b5 \u03af\u03b4\u03b9\u03b1 prompts, \u03af\u03b4\u03b9\u03b1 outputs \u03ba\u03b1\u03b9 \u03ba\u03bf\u03b9\u03bd\u03cc load profile. \u0397 \u03b1\u03c1\u03c7\u03ae \u03b5\u03af\u03bd\u03b1\u03b9 \u03af\u03b4\u03b9\u03b1 \u03bc\u03b5 \u03c4\u03b7\u03bd <a href=\"https:\/\/twodots.gr\/llm4llm-gpu-kernel-dokimi-pragmatiko-montelo\/\">end-to-end \u03b4\u03bf\u03ba\u03b9\u03bc\u03ae \u03b5\u03bd\u03cc\u03c2 \u03b3\u03c1\u03ae\u03b3\u03bf\u03c1\u03bf\u03c5 GPU kernel \u03bc\u03ad\u03c3\u03b1 \u03c3\u03c4\u03bf \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03bf<\/a>: \u03bc\u03b9\u03b1 \u03c4\u03bf\u03c0\u03b9\u03ba\u03ae \u03bc\u03ad\u03c4\u03c1\u03b7\u03c3\u03b7 \u03b4\u03b5\u03bd \u03b1\u03c1\u03ba\u03b5\u03af \u03b1\u03bd \u03c4\u03bf \u03c3\u03c5\u03bd\u03bf\u03bb\u03b9\u03ba\u03cc \u03c3\u03cd\u03c3\u03c4\u03b7\u03bc\u03b1 \u03b4\u03b5\u03bd \u03ba\u03b5\u03c1\u03b4\u03af\u03b6\u03b5\u03b9.<\/p>\n<p>\u03a4\u03b1 quality checks \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03ba\u03b1\u03bb\u03cd\u03c0\u03c4\u03bf\u03c5\u03bd \u03c0\u03b5\u03c1\u03b9\u03c3\u03c3\u03cc\u03c4\u03b5\u03c1\u03b1 \u03b1\u03c0\u03cc exact match. \u0393\u03b9\u03b1 \u03b5\u03bb\u03b5\u03cd\u03b8\u03b5\u03c1\u03b5\u03c2 \u03b1\u03c0\u03b1\u03bd\u03c4\u03ae\u03c3\u03b5\u03b9\u03c2 \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 output agreement, task metrics, groundedness, tool-call correctness \u03ba\u03b1\u03b9 \u03c7\u03b5\u03b9\u03c1\u03bf\u03ba\u03af\u03bd\u03b7\u03c4\u03b7 \u03b5\u03c0\u03b9\u03b8\u03b5\u03ce\u03c1\u03b7\u03c3\u03b7 failure clusters. \u03a4\u03bf approximate hit \u03b4\u03b5\u03bd \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03c0\u03b5\u03c1\u03bd\u03ac \u03b1\u03cc\u03c1\u03b1\u03c4\u03bf: \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 telemetry \u03b3\u03b9\u03b1 deviation, repair ratio \u03ba\u03b1\u03b9 fallback.<\/p>\n<p>\u03a4\u03bf hard memory budget \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03b1\u03c6\u03ae\u03bd\u03b5\u03b9 \u03c7\u03ce\u03c1\u03bf \u03b3\u03b9\u03b1 weights, \u03b5\u03bd\u03b5\u03c1\u03b3\u03ac requests \u03ba\u03b1\u03b9 generation KV cache. \u03a0\u03b1\u03c1\u03ac\u03bb\u03bb\u03b7\u03bb\u03b1, \u03b7 cache \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 tenant isolation, versioned keys \u03b3\u03b9\u03b1 model \u03ba\u03b1\u03b9 tokenizer, \u03ad\u03bb\u03b5\u03b3\u03c7\u03bf \u03b1\u03ba\u03cd\u03c1\u03c9\u03c3\u03b7\u03c2 \u03cc\u03c4\u03b1\u03bd \u03b1\u03bb\u03bb\u03ac\u03b6\u03bf\u03c5\u03bd prompts \u03ba\u03b1\u03b9 \u03c3\u03b1\u03c6\u03ae \u03c0\u03bf\u03bb\u03b9\u03c4\u03b9\u03ba\u03ae \u03b3\u03b9\u03b1 \u03b5\u03c5\u03b1\u03af\u03c3\u03b8\u03b7\u03c4\u03bf retrieved content. \u0397 \u03c4\u03b1\u03c7\u03cd\u03c4\u03b7\u03c4\u03b1 \u03b4\u03b5\u03bd \u03b4\u03b9\u03ba\u03b1\u03b9\u03bf\u03bb\u03bf\u03b3\u03b5\u03af cross-user leakage.<\/p>\n<div class=\"td-decision-band\">\n<p class=\"td-decision-kicker\">\u039a\u03b1\u03bd\u03cc\u03bd\u03b1\u03c2 \u03b1\u03c0\u03cc\u03c6\u03b1\u03c3\u03b7\u03c2 \u03b3\u03b9\u03b1 KVBoost<\/p>\n<p class=\"td-decision-title\">\u0395\u03bd\u03b5\u03c1\u03b3\u03bf\u03c0\u03bf\u03b9\u03ae\u03c3\u03c4\u03b5 approximate reuse \u03bc\u03cc\u03bd\u03bf \u03cc\u03c4\u03b1\u03bd \u03c4\u03b1 traces \u03b1\u03c0\u03bf\u03b4\u03b5\u03b9\u03ba\u03bd\u03cd\u03bf\u03c5\u03bd \u03c4\u03bf \u03ba\u03ad\u03c1\u03b4\u03bf\u03c2<\/p>\n<p>Go \u03c3\u03b7\u03bc\u03b1\u03af\u03bd\u03b5\u03b9 \u03c3\u03c5\u03c7\u03bd\u03ac \u03b5\u03c0\u03b1\u03bd\u03b1\u03bb\u03b1\u03bc\u03b2\u03b1\u03bd\u03cc\u03bc\u03b5\u03bd\u03b1 non-prefix chunks, \u03bc\u03b5\u03c4\u03c1\u03b7\u03bc\u03ad\u03bd\u03bf prefill bottleneck, \u03c3\u03c4\u03b1\u03b8\u03b5\u03c1\u03cc output agreement, \u03b5\u03bb\u03b5\u03b3\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf recompute ratio, tenant-safe keys \u03ba\u03b1\u03b9 \u03ac\u03bc\u03b5\u03c3\u03bf fallback. No-go \u03c3\u03b7\u03bc\u03b1\u03af\u03bd\u03b5\u03b9 \u03ba\u03c5\u03c1\u03af\u03c9\u03c2 \u03ba\u03bf\u03b9\u03bd\u03cc leading prefix, decode-bound latency, \u03b1\u03c3\u03c4\u03b1\u03b8\u03ae\u03c2 \u03c0\u03bf\u03b9\u03cc\u03c4\u03b7\u03c4\u03b1 \u03ae probe cost \u03c0\u03bf\u03c5 \u03b5\u03be\u03b1\u03c6\u03b1\u03bd\u03af\u03b6\u03b5\u03b9 \u03c4\u03bf \u03cc\u03c6\u03b5\u03bb\u03bf\u03c2.<\/p>\n<\/div>\n<p>\u0397 \u03c0\u03b1\u03c1\u03b1\u03b3\u03c9\u03b3\u03b9\u03ba\u03ae \u03b1\u03c0\u03cc\u03c6\u03b1\u03c3\u03b7 \u03bc\u03bf\u03b9\u03ac\u03b6\u03b5\u03b9 \u03c0\u03b5\u03c1\u03b9\u03c3\u03c3\u03cc\u03c4\u03b5\u03c1\u03bf \u03bc\u03b5 \u03b1\u03be\u03b9\u03bf\u03bb\u03cc\u03b3\u03b7\u03c3\u03b7 \u03c3\u03c5\u03c3\u03c4\u03ae\u03bc\u03b1\u03c4\u03bf\u03c2 \u03c0\u03b1\u03c1\u03ac \u03bc\u03b5 \u03b5\u03b3\u03ba\u03b1\u03c4\u03ac\u03c3\u03c4\u03b1\u03c3\u03b7 \u03b2\u03b9\u03b2\u03bb\u03b9\u03bf\u03b8\u03ae\u03ba\u03b7\u03c2. \u03a7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03c0\u03b1\u03c1\u03b1\u03ba\u03bf\u03bb\u03bf\u03cd\u03b8\u03b7\u03c3\u03b7 p95 \u03ba\u03b1\u03b9 p99, \u03cc\u03c7\u03b9 \u03bc\u03cc\u03bd\u03bf mean TTFT, \u03ba\u03b1\u03b8\u03ce\u03c2 \u03ba\u03b1\u03b9 \u03be\u03b5\u03c7\u03c9\u03c1\u03b9\u03c3\u03c4\u03ad\u03c2 \u03ba\u03b1\u03bc\u03c0\u03cd\u03bb\u03b5\u03c2 \u03b1\u03bd\u03ac context bucket, cold \u03ae warm \u03ba\u03b1\u03c4\u03ac\u03c3\u03c4\u03b1\u03c3\u03b7 \u03ba\u03b1\u03b9 \u03c4\u03cd\u03c0\u03bf cache hit.<\/p>\n<h2 id=\"plaisio-apofasis\">\u0395\u03c0\u03c4\u03ac \u03ad\u03bb\u03b5\u03b3\u03c7\u03bf\u03b9 \u03c0\u03c1\u03b9\u03bd \u03b1\u03c0\u03cc \u03c4\u03b7\u03bd \u03b1\u03c0\u03cc\u03c6\u03b1\u03c3\u03b7<\/h2>\n<p>\u0388\u03bd\u03b1 pilot \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03c0\u03b1\u03c1\u03b1\u03bc\u03b5\u03af\u03bd\u03b5\u03b9 \u03bc\u03b9\u03ba\u03c1\u03cc, \u03b1\u03c1\u03ba\u03b5\u03af \u03bd\u03b1 \u03b1\u03bd\u03b1\u03c0\u03b1\u03c1\u03b9\u03c3\u03c4\u03ac \u03c4\u03bf \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc prompt topology \u03ba\u03b1\u03b9 \u03bd\u03b1 \u03ad\u03c7\u03b5\u03b9 \u03c3\u03b1\u03c6\u03ae acceptance criteria. \u039f\u03b9 \u03c0\u03b1\u03c1\u03b1\u03ba\u03ac\u03c4\u03c9 \u03ad\u03bb\u03b5\u03b3\u03c7\u03bf\u03b9 \u03bc\u03b5\u03c4\u03b1\u03c4\u03c1\u03ad\u03c0\u03bf\u03c5\u03bd \u03c4\u03bf KVBoost \u03b1\u03c0\u03cc \u03b8\u03b5\u03c9\u03c1\u03b7\u03c4\u03b9\u03ba\u03ae \u03b2\u03b5\u03bb\u03c4\u03b9\u03c3\u03c4\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b7 \u03c3\u03b5 \u03bc\u03b5\u03c4\u03c1\u03ae\u03c3\u03b9\u03bc\u03b7 \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03b7\u03c3\u03b9\u03b1\u03ba\u03ae \u03b5\u03c0\u03b9\u03bb\u03bf\u03b3\u03ae.<\/p>\n<div class=\"td-step-list\">\n<p class=\"td-step-list-title\">\u0395\u03c0\u03c4\u03ac \u03ad\u03bb\u03b5\u03b3\u03c7\u03bf\u03b9 \u03b3\u03b9\u03b1 KV cache reuse \u03c0\u03c1\u03b9\u03bd \u03b1\u03c0\u03cc \u03c4\u03bf production<\/p>\n<ol>\n<li><span class=\"td-step-kicker\">\u0421\u0442\u044a\u043f\u043a\u0430 1<\/span><strong>\u03a7\u03b1\u03c1\u03c4\u03bf\u03b3\u03c1\u03b1\u03c6\u03ae\u03c3\u03c4\u03b5 \u03c4\u03bf prompt topology<\/strong>\n<p>\u039c\u03b5\u03c4\u03c1\u03ae\u03c3\u03c4\u03b5 \u03c0\u03cc\u03c3\u03bf \u03ba\u03b5\u03af\u03bc\u03b5\u03bd\u03bf \u03b5\u03c0\u03b1\u03bd\u03b1\u03bb\u03b1\u03bc\u03b2\u03ac\u03bd\u03b5\u03c4\u03b1\u03b9, \u03b1\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 leading prefix \u03ae \u03bc\u03b5\u03c4\u03b1\u03ba\u03b9\u03bd\u03bf\u03cd\u03bc\u03b5\u03bd\u03bf chunk \u03ba\u03b1\u03b9 \u03c0\u03bf\u03b9\u03b1 \u03c4\u03bc\u03ae\u03bc\u03b1\u03c4\u03b1 \u03b1\u03bb\u03bb\u03ac\u03b6\u03bf\u03c5\u03bd \u03b1\u03bd\u03ac \u03c7\u03c1\u03ae\u03c3\u03c4\u03b7 \u03ae \u03b1\u03af\u03c4\u03b7\u03bc\u03b1.<\/p>\n<\/li>\n<li><span class=\"td-step-kicker\">\u0421\u0442\u044a\u043f\u043a\u0430 2<\/span><strong>\u03a7\u03c9\u03c1\u03af\u03c3\u03c4\u03b5 cold \u03ba\u03b1\u03b9 warm traffic<\/strong>\n<p>\u039a\u03b1\u03c4\u03b1\u03b3\u03c1\u03ac\u03c8\u03c4\u03b5 \u03be\u03b5\u03c7\u03c9\u03c1\u03b9\u03c3\u03c4\u03ac TTFT, hit rate \u03ba\u03b1\u03b9 reuse ratio, \u03ce\u03c3\u03c4\u03b5 \u03bf\u03b9 \u03b3\u03c1\u03ae\u03b3\u03bf\u03c1\u03b5\u03c2 warm \u03ba\u03bb\u03ae\u03c3\u03b5\u03b9\u03c2 \u03bd\u03b1 \u03bc\u03b7 \u03c3\u03c5\u03b3\u03ba\u03b1\u03bb\u03cd\u03c0\u03c4\u03bf\u03c5\u03bd \u03c4\u03bf \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc \u03ba\u03cc\u03c3\u03c4\u03bf\u03c2 \u03c4\u03c9\u03bd cold requests.<\/p>\n<\/li>\n<li><span class=\"td-step-kicker\">\u0421\u0442\u044a\u043f\u043a\u0430 3<\/span><strong>\u039a\u03bb\u03b5\u03b9\u03b4\u03ce\u03c3\u03c4\u03b5 \u03c4\u03b1 baselines<\/strong>\n<p>\u03a3\u03c5\u03b3\u03ba\u03c1\u03af\u03bd\u03b5\u03c4\u03b5 full recomputation, \u03c4\u03bf \u03c5\u03c0\u03ac\u03c1\u03c7\u03bf\u03bd prefix cache \u03ba\u03b1\u03b9 KVBoost \u03bc\u03b5 \u03af\u03b4\u03b9\u03bf model commit, tokenizer, prompt, output length \u03ba\u03b1\u03b9 concurrency.<\/p>\n<\/li>\n<li><span class=\"td-step-kicker\">\u0421\u0442\u044a\u043f\u043a\u0430 4<\/span><strong>\u039e\u03b5\u03ba\u03b9\u03bd\u03ae\u03c3\u03c4\u03b5 \u03bc\u03b5 exact-only reuse<\/strong>\n<p>\u0395\u03c0\u03b9\u03b2\u03b5\u03b2\u03b1\u03b9\u03ce\u03c3\u03c4\u03b5 \u03c0\u03c1\u03ce\u03c4\u03b1 \u03c4\u03bf prefix-hash path \u03ba\u03b1\u03b9 \u03c0\u03c1\u03bf\u03c3\u03b8\u03ad\u03c3\u03c4\u03b5 content-hash hits \u03bc\u03cc\u03bd\u03bf \u03cc\u03c4\u03b1\u03bd \u03c5\u03c0\u03ac\u03c1\u03c7\u03b5\u03b9 \u03bc\u03b5\u03c4\u03c1\u03ae\u03c3\u03b9\u03bc\u03bf \u03ba\u03ad\u03c1\u03b4\u03bf\u03c2 \u03ba\u03b1\u03b9 \u03b1\u03be\u03b9\u03cc\u03c0\u03b9\u03c3\u03c4\u03b7 repair telemetry.<\/p>\n<\/li>\n<li><span class=\"td-step-kicker\">\u0421\u0442\u044a\u043f\u043a\u0430 5<\/span><strong>\u039c\u03b5\u03c4\u03c1\u03ae\u03c3\u03c4\u03b5 \u03c0\u03bf\u03b9\u03cc\u03c4\u03b7\u03c4\u03b1 \u03b1\u03bd\u03ac hit type<\/strong>\n<p>\u03a3\u03c5\u03b3\u03ba\u03c1\u03af\u03bd\u03b5\u03c4\u03b5 exact, approximate \u03ba\u03b1\u03b9 miss \u03c3\u03b5 output agreement, task accuracy, groundedness \u03ba\u03b1\u03b9 tool-call correctness, \u03cc\u03c7\u03b9 \u03bc\u03cc\u03bd\u03bf \u03c3\u03b5 \u03c3\u03c5\u03bd\u03bf\u03bb\u03b9\u03ba\u03cc \u03bc\u03ad\u03c3\u03bf \u03cc\u03c1\u03bf.<\/p>\n<\/li>\n<li><span class=\"td-step-kicker\">\u0421\u0442\u044a\u043f\u043a\u0430 6<\/span><strong>\u03a0\u03c1\u03bf\u03c3\u03c4\u03b1\u03c4\u03ad\u03c8\u03c4\u03b5 \u03bc\u03bd\u03ae\u03bc\u03b7 \u03ba\u03b1\u03b9 \u03b4\u03b5\u03b4\u03bf\u03bc\u03ad\u03bd\u03b1<\/strong>\n<p>\u039f\u03c1\u03af\u03c3\u03c4\u03b5 hard cache budget, eviction, model-version keys, tenant isolation \u03ba\u03b1\u03b9 invalidation \u03cc\u03c4\u03b1\u03bd \u03b1\u03bb\u03bb\u03ac\u03b6\u03b5\u03b9 tokenizer, policy \u03ae retrieved document.<\/p>\n<\/li>\n<li><span class=\"td-step-kicker\">\u0421\u0442\u044a\u043f\u043a\u0430 7<\/span><strong>\u039a\u03c1\u03b1\u03c4\u03ae\u03c3\u03c4\u03b5 fallback \u03ba\u03b1\u03b9 observability<\/strong>\n<p>\u0395\u03bd\u03b5\u03c1\u03b3\u03bf\u03c0\u03bf\u03b9\u03ae\u03c3\u03c4\u03b5 \u03ac\u03bc\u03b5\u03c3\u03b7 \u03b5\u03c0\u03b9\u03c3\u03c4\u03c1\u03bf\u03c6\u03ae \u03c3\u03b5 full recomputation \u03ae prefix caching \u03ba\u03b1\u03b9 \u03c0\u03b1\u03c1\u03b1\u03ba\u03bf\u03bb\u03bf\u03c5\u03b8\u03ae\u03c3\u03c4\u03b5 p95\/p99 TTFT, repair ratio, disk promotions \u03ba\u03b1\u03b9 quality alerts.<\/p>\n<\/li>\n<\/ol>\n<\/div>\n<p>\u03a4\u03bf KVBoost \u03b4\u03b5\u03af\u03c7\u03bd\u03b5\u03b9 \u03cc\u03c4\u03b9 \u03b7 \u03b8\u03ad\u03c3\u03b7 \u03c4\u03bf\u03c5 \u03ba\u03bf\u03b9\u03bd\u03bf\u03cd \u03c0\u03b5\u03c1\u03b9\u03b5\u03c7\u03bf\u03bc\u03ad\u03bd\u03bf\u03c5 \u03b4\u03b5\u03bd \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03bd\u03b1 \u03b1\u03ba\u03c5\u03c1\u03ce\u03bd\u03b5\u03b9 \u03c4\u03b7\u03bd \u03b5\u03c0\u03b1\u03bd\u03b1\u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b7. \u0397 \u03b1\u03be\u03af\u03b1 \u03c4\u03bf\u03c5 \u03cc\u03bc\u03c9\u03c2 \u03b1\u03c0\u03bf\u03b4\u03b5\u03b9\u03ba\u03bd\u03cd\u03b5\u03c4\u03b1\u03b9 \u03bc\u03cc\u03bd\u03bf \u03b1\u03bd \u03c7\u03b1\u03bc\u03b7\u03bb\u03ce\u03bd\u03b5\u03b9 \u03c4\u03bf \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc prefill latency, \u03b4\u03b9\u03b1\u03c4\u03b7\u03c1\u03b5\u03af \u03c4\u03b7\u03bd \u03c0\u03bf\u03b9\u03cc\u03c4\u03b7\u03c4\u03b1 \u03c3\u03c4\u03bf \u03b4\u03b9\u03ba\u03cc \u03c3\u03b1\u03c2 workload \u03ba\u03b1\u03b9 \u03bb\u03b5\u03b9\u03c4\u03bf\u03c5\u03c1\u03b3\u03b5\u03af \u03bc\u03ad\u03c3\u03b1 \u03c3\u03b5 \u03b5\u03bb\u03b5\u03b3\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf memory, security \u03ba\u03b1\u03b9 rollback \u03c0\u03bb\u03b1\u03af\u03c3\u03b9\u03bf.<\/p>\n<section class=\"td-service-cta\">\n<div class=\"td-service-cta-content\">\n<p class=\"td-service-cta-kicker\">\u0391\u03c0\u03cc \u03c4\u03bf LLM benchmark \u03c3\u03b5 \u03bc\u03b5\u03c4\u03c1\u03ae\u03c3\u03b9\u03bc\u03bf AI workflow<\/p>\n<p class=\"td-service-cta-title\">\u03a3\u03c7\u03b5\u03b4\u03b9\u03ac\u03c3\u03c4\u03b5 latency, \u03c0\u03bf\u03b9\u03cc\u03c4\u03b7\u03c4\u03b1 \u03ba\u03b1\u03b9 fallback \u03bc\u03b1\u03b6\u03af<\/p>\n<p>\u0397 TWO DOTS \u03c7\u03b1\u03c1\u03c4\u03bf\u03b3\u03c1\u03b1\u03c6\u03b5\u03af \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03ac prompt traces, RAG \u03ba\u03b1\u03b9 agent \u03c1\u03bf\u03ad\u03c2, acceptance criteria, observability \u03ba\u03b1\u03b9 \u03b1\u03c3\u03c6\u03b1\u03bb\u03ae integrations \u03ce\u03c3\u03c4\u03b5 \u03bc\u03b9\u03b1 inference \u03b2\u03b5\u03bb\u03c4\u03b9\u03c3\u03c4\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b7 \u03bd\u03b1 \u03c0\u03b1\u03c1\u03ac\u03b3\u03b5\u03b9 \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03b7\u03c3\u03b9\u03b1\u03ba\u03cc \u03b1\u03c0\u03bf\u03c4\u03ad\u03bb\u03b5\u03c3\u03bc\u03b1 \u03c7\u03c9\u03c1\u03af\u03c2 \u03bd\u03b1 \u03ba\u03c1\u03cd\u03b2\u03b5\u03b9 \u03b1\u03c0\u03bf\u03ba\u03bb\u03af\u03c3\u03b5\u03b9\u03c2 \u03c0\u03bf\u03b9\u03cc\u03c4\u03b7\u03c4\u03b1\u03c2.<\/p>\n<div class=\"td-service-cta-actions\"><a class=\"td-service-cta-button\" href=\"https:\/\/twodots.gr\/aftomatismoi-epicheiriseon-ai\/\">\u03a3\u03c5\u03b6\u03b7\u03c4\u03ae\u03c3\u03c4\u03b5 \u03ad\u03bd\u03b1\u03bd \u03b5\u03bb\u03b5\u03b3\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf AI \u03b1\u03c5\u03c4\u03bf\u03bc\u03b1\u03c4\u03b9\u03c3\u03bc\u03cc<\/a><\/div>\n<\/div>\n<\/section>\n<section id=\"sychnes-erotiseis\" class=\"td-faq-section\">\n<div class=\"td-faq\">\n<p class=\"td-faq-heading\">\u0427\u0435\u0441\u0442\u043e \u0437\u0430\u0434\u0430\u0432\u0430\u043d\u0438 \u0432\u044a\u043f\u0440\u043e\u0441\u0438<\/p>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u03a4\u03b9 \u03b5\u03af\u03bd\u03b1\u03b9 \u03c4\u03bf KVBoost;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u0395\u03af\u03bd\u03b1\u03b9 \u03b5\u03c1\u03b5\u03c5\u03bd\u03b7\u03c4\u03b9\u03ba\u03cc \u03c3\u03cd\u03c3\u03c4\u03b7\u03bc\u03b1 chunk-level \u03b5\u03c0\u03b1\u03bd\u03b1\u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b7\u03c2 KV cache \u03b3\u03b9\u03b1 Hugging Face decoder models \u03bc\u03b5 RoPE. \u03a3\u03c5\u03bd\u03b4\u03c5\u03ac\u03b6\u03b5\u03b9 exact \u03ba\u03b1\u03b9 approximate cache lookup \u03bc\u03b5 \u03c5\u03c0\u03bf\u03c7\u03c1\u03b5\u03c9\u03c4\u03b9\u03ba\u03ae \u03b5\u03c0\u03b9\u03c3\u03ba\u03b5\u03c5\u03ae \u03cc\u03c4\u03b1\u03bd \u03c4\u03bf \u03af\u03b4\u03b9\u03bf \u03c0\u03b5\u03c1\u03b9\u03b5\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf \u03b5\u03bc\u03c6\u03b1\u03bd\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 \u03c3\u03b5 \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03b5\u03c4\u03b9\u03ba\u03ae \u03b8\u03ad\u03c3\u03b7 \u03ae context.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u03a0\u03ce\u03c2 \u03b4\u03b9\u03b1\u03c6\u03ad\u03c1\u03b5\u03b9 \u03b1\u03c0\u03cc \u03c4\u03bf prefix caching;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u03a4\u03bf prefix caching \u03b1\u03be\u03b9\u03bf\u03c0\u03bf\u03b9\u03b5\u03af \u03ba\u03bf\u03b9\u03bd\u03ae \u03b1\u03c1\u03c7\u03b9\u03ba\u03ae \u03b1\u03ba\u03bf\u03bb\u03bf\u03c5\u03b8\u03af\u03b1 tokens. \u03a4\u03bf KVBoost \u03b1\u03bd\u03b1\u03b6\u03b7\u03c4\u03ac \u03b5\u03c0\u03b1\u03bd\u03b1\u03bb\u03b1\u03bc\u03b2\u03b1\u03bd\u03cc\u03bc\u03b5\u03bd\u03b1 chunks \u03ba\u03b1\u03b9 \u03c3\u03b5 \u03bc\u03b7 \u03b1\u03c1\u03c7\u03b9\u03ba\u03ad\u03c2 \u03b8\u03ad\u03c3\u03b5\u03b9\u03c2, \u03b1\u03bb\u03bb\u03ac \u03c0\u03bb\u03b7\u03c1\u03ce\u03bd\u03b5\u03b9 \u03c0\u03c1\u03cc\u03c3\u03b8\u03b5\u03c4\u03bf \u03ba\u03cc\u03c3\u03c4\u03bf\u03c2 \u03b3\u03b9\u03b1 \u03bd\u03b1 \u03b4\u03b9\u03bf\u03c1\u03b8\u03ce\u03c3\u03b5\u03b9 positional \u03ba\u03b1\u03b9 context \u03b1\u03c0\u03bf\u03ba\u03bb\u03af\u03c3\u03b5\u03b9\u03c2.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u03a4\u03b9 \u03b5\u03af\u03bd\u03b1\u03b9 \u03c4\u03bf seam error;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u0395\u03af\u03bd\u03b1\u03b9 \u03b7 \u03b1\u03c0\u03cc\u03ba\u03bb\u03b9\u03c3\u03b7 \u03c3\u03c4\u03b1 \u03cc\u03c1\u03b9\u03b1 \u03cc\u03c4\u03b1\u03bd \u03b5\u03bd\u03ce\u03bd\u03bf\u03bd\u03c4\u03b1\u03b9 cached chunks \u03c0\u03bf\u03c5 \u03b5\u03af\u03c7\u03b1\u03bd \u03c5\u03c0\u03bf\u03bb\u03bf\u03b3\u03b9\u03c3\u03c4\u03b5\u03af \u03bc\u03b5 \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03b5\u03c4\u03b9\u03ba\u03cc \u03c0\u03c1\u03bf\u03b7\u03b3\u03bf\u03cd\u03bc\u03b5\u03bd\u03bf context. \u0391\u03bd \u03b4\u03b5\u03bd \u03b5\u03c0\u03b9\u03c3\u03ba\u03b5\u03c5\u03b1\u03c3\u03c4\u03b5\u03af, \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b1\u03bb\u03bb\u03ac\u03be\u03b5\u03b9 \u03c4\u03b9\u03c2 attention \u03b1\u03bd\u03b1\u03c0\u03b1\u03c1\u03b1\u03c3\u03c4\u03ac\u03c3\u03b5\u03b9\u03c2 \u03ba\u03b1\u03b9 \u03c4\u03bf \u03b1\u03c0\u03bf\u03c4\u03ad\u03bb\u03b5\u03c3\u03bc\u03b1.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u03a0\u03bf\u03b9\u03b1 \u03b5\u03c0\u03af\u03b4\u03bf\u03c3\u03b7 \u03b1\u03bd\u03b1\u03c6\u03ad\u03c1\u03b5\u03b9 \u03b7 \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u03a3\u03c4\u03bf benchmark 1.000 \u03b4\u03b5\u03b9\u03b3\u03bc\u03ac\u03c4\u03c9\u03bd bug localization \u03bc\u03b5 Qwen2.5-3B \u03ba\u03b1\u03b9 RTX 4060, \u03c4\u03bf KVBoost \u03b5\u03af\u03c7\u03b5 \u03bc\u03ad\u03c3\u03bf TTFT 142,4 ms, \u03ad\u03bd\u03b1\u03bd\u03c4\u03b9 639,1 ms \u03b3\u03b9\u03b1 full recomputation \u03ba\u03b1\u03b9 165,5 ms \u03b3\u03b9\u03b1 vLLM prefix caching.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u03a5\u03c0\u03ae\u03c1\u03be\u03b5 \u03b1\u03c0\u03ce\u03bb\u03b5\u03b9\u03b1 \u03b1\u03ba\u03c1\u03af\u03b2\u03b5\u03b9\u03b1\u03c2;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u0394\u03b5\u03bd \u03b1\u03bd\u03b9\u03c7\u03bd\u03b5\u03cd\u03c4\u03b7\u03ba\u03b5 \u03c3\u03c4\u03bf \u03c3\u03c5\u03b3\u03ba\u03b5\u03ba\u03c1\u03b9\u03bc\u03ad\u03bd\u03bf multiple-choice task: 99,2% exact match \u03b3\u03b9\u03b1 KVBoost \u03ad\u03bd\u03b1\u03bd\u03c4\u03b9 99,1% \u03b3\u03b9\u03b1 \u03c4\u03b9\u03c2 \u03b4\u03cd\u03bf \u03b2\u03ac\u03c3\u03b5\u03b9\u03c2 \u03c3\u03cd\u03b3\u03ba\u03c1\u03b9\u03c3\u03b7\u03c2. \u03a4\u03bf \u03b1\u03c0\u03bf\u03c4\u03ad\u03bb\u03b5\u03c3\u03bc\u03b1 \u03b4\u03b5\u03bd \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03b5\u03af \u03b3\u03b5\u03bd\u03b9\u03ba\u03ae \u03b5\u03b3\u03b3\u03cd\u03b7\u03c3\u03b7 \u03b3\u03b9\u03b1 \u03ac\u03bb\u03bb\u03b1 \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03b1 \u03ae workloads.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u039c\u03b5\u03b9\u03ce\u03bd\u03b5\u03b9 \u03c3\u03b7\u03bc\u03b1\u03bd\u03c4\u03b9\u03ba\u03ac \u03c4\u03b7 \u03bc\u03ad\u03b3\u03b9\u03c3\u03c4\u03b7 GPU \u03bc\u03bd\u03ae\u03bc\u03b7;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u038c\u03c7\u03b9 \u03c3\u03c4\u03bf \u03c0\u03b5\u03af\u03c1\u03b1\u03bc\u03b1 \u03c4\u03b7\u03c2 \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1\u03c2. \u0397 peak GPU allocation \u03bc\u03b5\u03b9\u03ce\u03b8\u03b7\u03ba\u03b5 \u03ba\u03b1\u03c4\u03ac 14,8 MB \u03ae 0,24%, \u03b5\u03c0\u03bf\u03bc\u03ad\u03bd\u03c9\u03c2 \u03c4\u03bf \u03ba\u03cd\u03c1\u03b9\u03bf \u03bc\u03b5\u03c4\u03c1\u03b7\u03bc\u03ad\u03bd\u03bf \u03cc\u03c6\u03b5\u03bb\u03bf\u03c2 \u03ae\u03c4\u03b1\u03bd \u03c7\u03b1\u03bc\u03b7\u03bb\u03cc\u03c4\u03b5\u03c1\u03bf prefill latency \u03ba\u03b1\u03b9 \u03cc\u03c7\u03b9 \u03bc\u03b5\u03b3\u03ac\u03bb\u03b7 \u03c0\u03c4\u03ce\u03c3\u03b7 \u03c4\u03b7\u03c2 \u03ba\u03bf\u03c1\u03c5\u03c6\u03b1\u03af\u03b1\u03c2 \u03bc\u03bd\u03ae\u03bc\u03b7\u03c2.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u03a0\u03cc\u03c4\u03b5 \u03b5\u03af\u03bd\u03b1\u03b9 \u03c0\u03c1\u03bf\u03c4\u03b9\u03bc\u03cc\u03c4\u03b5\u03c1\u03bf \u03c4\u03bf \u03b1\u03c0\u03bb\u03cc prefix cache;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u038c\u03c4\u03b1\u03bd \u03c4\u03b1 \u03b1\u03b9\u03c4\u03ae\u03bc\u03b1\u03c4\u03b1 \u03bc\u03bf\u03b9\u03c1\u03ac\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc leading prefix \u03ba\u03b1\u03b9 \u03b1\u03c0\u03b1\u03b9\u03c4\u03b5\u03af\u03c4\u03b1\u03b9 \u03b1\u03ba\u03c1\u03b9\u03b2\u03ae\u03c2 positional correctness, \u03c4\u03bf prefix cache \u03b1\u03c0\u03bf\u03c6\u03b5\u03cd\u03b3\u03b5\u03b9 seam-repair overhead. \u03a4\u03bf KVBoost \u03ad\u03c7\u03b5\u03b9 \u03c0\u03b5\u03c1\u03b9\u03c3\u03c3\u03cc\u03c4\u03b5\u03c1\u03bf \u03bd\u03cc\u03b7\u03bc\u03b1 \u03cc\u03c4\u03b1\u03bd \u03c4\u03b1 \u03ba\u03bf\u03b9\u03bd\u03ac chunks \u03bc\u03b5\u03c4\u03b1\u03ba\u03b9\u03bd\u03bf\u03cd\u03bd\u03c4\u03b1\u03b9 \u03ae \u03c0\u03b1\u03c1\u03b5\u03bc\u03b2\u03ac\u03bb\u03bb\u03bf\u03bd\u03c4\u03b1\u03b9.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u03a4\u03b9 \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03bc\u03b5\u03c4\u03c1\u03ae\u03c3\u03b5\u03b9 \u03bc\u03b9\u03b1 \u03bf\u03bc\u03ac\u03b4\u03b1 \u03c0\u03c1\u03b9\u03bd \u03c4\u03bf production rollout;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u03a0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03bc\u03b5\u03c4\u03c1\u03ae\u03c3\u03b5\u03b9 cold \u03ba\u03b1\u03b9 warm TTFT, p95 \u03ba\u03b1\u03b9 p99, cache hit \u03ba\u03b1\u03b9 reuse ratio, recompute ratio, output agreement, memory budget \u03ba\u03b1\u03b9 fallback rate \u03c0\u03ac\u03bd\u03c9 \u03c3\u03b5 \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03ac prompts. \u039f\u03b9 \u03bc\u03b5\u03c4\u03c1\u03ae\u03c3\u03b5\u03b9\u03c2 \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03b3\u03af\u03bd\u03bf\u03c5\u03bd \u03c3\u03c4\u03bf \u03b4\u03b9\u03ba\u03cc \u03c4\u03b7\u03c2 serving stack \u03ba\u03b1\u03b9 hardware.<\/p>\n<\/div>\n<\/details>\n<\/div>\n<\/section>\n<div class=\"td-source-list\">\n<p id=\"piges\" class=\"td-source-list-title\">\u0418\u0437\u0442\u043e\u0447\u043d\u0438\u0446\u0438<\/p>\n<ul>\n<li><a href=\"https:\/\/arxiv.org\/abs\/2608.21362\" target=\"_blank\" rel=\"noopener\">arXiv \u2014 KVBoost: Chunk-Level Key-Value Cache Reuse with Deviation-Guided Recomputation<\/a><\/li>\n<li><a href=\"https:\/\/arxiv.org\/html\/2608.21362\" target=\"_blank\" rel=\"noopener\">arXiv HTML \u2014 \u03c0\u03bb\u03ae\u03c1\u03b7\u03c2 \u03b1\u03c1\u03c7\u03b9\u03c4\u03b5\u03ba\u03c4\u03bf\u03bd\u03b9\u03ba\u03ae, \u03c0\u03b5\u03b9\u03c1\u03ac\u03bc\u03b1\u03c4\u03b1, ablations \u03ba\u03b1\u03b9 \u03c0\u03b5\u03c1\u03b9\u03bf\u03c1\u03b9\u03c3\u03bc\u03bf\u03af \u03c4\u03bf\u03c5 KVBoost<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/pythongiant\/kvboost\" target=\"_blank\" rel=\"noopener\">\u0395\u03c0\u03af\u03c3\u03b7\u03bc\u03bf KVBoost repository \u2014 \u03ba\u03ce\u03b4\u03b9\u03ba\u03b1\u03c2, \u03b5\u03b3\u03ba\u03b1\u03c4\u03ac\u03c3\u03c4\u03b1\u03c3\u03b7, benchmarks \u03ba\u03b1\u03b9 \u03c4\u03c1\u03ad\u03c7\u03bf\u03c5\u03c3\u03b5\u03c2 \u03b4\u03c5\u03bd\u03b1\u03c4\u03cc\u03c4\u03b7\u03c4\u03b5\u03c2<\/a><\/li>\n<li><a href=\"https:\/\/docs.vllm.ai\/en\/stable\/features\/automatic_prefix_caching\/\" target=\"_blank\" rel=\"noopener\">vLLM Documentation \u2014 Automatic Prefix Caching, \u03c7\u03c1\u03ae\u03c3\u03b5\u03b9\u03c2 \u03ba\u03b1\u03b9 \u03cc\u03c1\u03b9\u03b1<\/a><\/li>\n<li><a href=\"https:\/\/www.nist.gov\/itl\/ai-risk-management-framework\" target=\"_blank\" rel=\"noopener\">NIST AI Risk Management Framework \u2014 \u03bc\u03ad\u03c4\u03c1\u03b7\u03c3\u03b7, \u03b4\u03b9\u03b1\u03c7\u03b5\u03af\u03c1\u03b9\u03c3\u03b7 \u03ba\u03b9\u03bd\u03b4\u03cd\u03bd\u03bf\u03c5 \u03ba\u03b1\u03b9 \u03b1\u03bd\u03b8\u03c1\u03ce\u03c0\u03b9\u03bd\u03b7 \u03b5\u03c0\u03bf\u03c0\u03c4\u03b5\u03af\u03b1<\/a><\/li>\n<\/ul>\n<\/div>","protected":false},"excerpt":{"rendered":"<p>\u03a4\u03bf KVBoost \u03b5\u03c0\u03b1\u03bd\u03b1\u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03b5\u03af chunks \u03c4\u03b7\u03c2 KV cache \u03c0\u03ad\u03c1\u03b1 \u03b1\u03c0\u03cc \u03c4\u03bf \u03ba\u03bf\u03b9\u03bd\u03cc prefix, \u03bc\u03b5 seam repair \u03ba\u03b1\u03b9 \u03b5\u03bb\u03b5\u03b3\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf memory budget.<\/p>","protected":false},"author":1,"featured_media":98921,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_gspb_post_css":"","content-type":"","footnotes":""},"categories":[199],"tags":[8988,17819,20246,19427,8233],"class_list":["post-97841","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-techniti-noimosyni","tag-ai-infrastructure","tag-hugging-face","tag-kv-cache","tag-llm-inference","tag-rag"],"blocksy_meta":{"styles_descriptor":{"styles":{"desktop":"","tablet":"","mobile":""},"google_fonts":[],"version":8}},"_links":{"self":[{"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/posts\/97841","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/comments?post=97841"}],"version-history":[{"count":1,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/posts\/97841\/revisions"}],"predecessor-version":[{"id":98922,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/posts\/97841\/revisions\/98922"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/media\/98921"}],"wp:attachment":[{"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/media?parent=97841"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/categories?post=97841"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/tags?post=97841"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}