{"id":87777,"date":"2026-08-03T12:48:21","date_gmt":"2026-08-03T09:48:21","guid":{"rendered":"https:\/\/twodots.gr\/?p=87777"},"modified":"2026-08-03T12:48:22","modified_gmt":"2026-08-03T09:48:22","slug":"moonep-exisorropisi-ai-experts-moe","status":"publish","type":"post","link":"https:\/\/twodots.gr\/bg\/moonep-exisorropisi-ai-experts-moe\/","title":{"rendered":"MoonEP: \u03c0\u03ce\u03c2 \u03b7 \u03b5\u03be\u03b9\u03c3\u03bf\u03c1\u03c1\u03cc\u03c0\u03b7\u03c3\u03b7 \u03c4\u03c9\u03bd AI experts \u03b1\u03bb\u03bb\u03ac\u03b6\u03b5\u03b9 \u03c4\u03b7\u03bd \u03c5\u03c0\u03bf\u03b4\u03bf\u03bc\u03ae \u03c4\u03c9\u03bd MoE \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03c9\u03bd"},"content":{"rendered":"<div class=\"td-article-lede\">\n<p><strong>\u03a3\u03cd\u03bd\u03c4\u03bf\u03bc\u03b7 \u03b1\u03c0\u03ac\u03bd\u03c4\u03b7\u03c3\u03b7:<\/strong> \u03c4\u03bf MoonEP \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03b5\u03af \u03bd\u03b1 \u03ba\u03ac\u03bd\u03b5\u03b9 \u03c0\u03c1\u03bf\u03b2\u03bb\u03ad\u03c8\u03b9\u03bc\u03bf \u03c4\u03bf expert parallelism \u03c3\u03c4\u03b1 Mixture-of-Experts \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03b1. \u039c\u03b5 online \u03c3\u03c7\u03b5\u03b4\u03b9\u03b1\u03c3\u03bc\u03cc redundant experts, \u03c0\u03c1\u03bf\u03c6\u03cc\u03c1\u03c4\u03c9\u03c3\u03b7 \u03b2\u03b1\u03c1\u03ce\u03bd, zero-copy \u03b5\u03c0\u03b9\u03ba\u03bf\u03b9\u03bd\u03c9\u03bd\u03af\u03b1 \u03ba\u03b1\u03b9 \u03c3\u03c4\u03b1\u03c4\u03b9\u03ba\u03ac shapes, \u03b4\u03b7\u03bb\u03ce\u03bd\u03b5\u03b9 \u03cc\u03c4\u03b9 \u03ba\u03ac\u03b8\u03b5 rank \u03b5\u03c0\u03b5\u03be\u03b5\u03c1\u03b3\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03b1\u03ba\u03c1\u03b9\u03b2\u03ce\u03c2 S \u00d7 K tokens \u03b1\u03ba\u03cc\u03bc\u03b7 \u03ba\u03b1\u03b9 \u03cc\u03c4\u03b1\u03bd \u03bf router \u03ba\u03b1\u03c4\u03b1\u03bd\u03ad\u03bc\u03b5\u03b9 \u03ac\u03bd\u03b9\u03c3\u03b1 \u03c4\u03bf \u03c6\u03bf\u03c1\u03c4\u03af\u03bf.<\/p>\n<p>\u0397 \u03c5\u03c0\u03cc\u03c3\u03c7\u03b5\u03c3\u03b7 \u03b5\u03af\u03bd\u03b1\u03b9 \u03c4\u03b5\u03c7\u03bd\u03b9\u03ba\u03ac \u03c3\u03b7\u03bc\u03b1\u03bd\u03c4\u03b9\u03ba\u03ae, \u03b5\u03c0\u03b5\u03b9\u03b4\u03ae \u03ad\u03bd\u03b1 MoE training step \u03c0\u03b5\u03c1\u03b9\u03bc\u03ad\u03bd\u03b5\u03b9 \u03c4\u03bf\u03bd \u03c0\u03b9\u03bf \u03c6\u03bf\u03c1\u03c4\u03c9\u03bc\u03ad\u03bd\u03bf rank. \u0391\u03bd \u03c4\u03bf \u03c6\u03bf\u03c1\u03c4\u03af\u03bf \u03bc\u03b5\u03c4\u03b1\u03b2\u03ac\u03bb\u03bb\u03b5\u03c4\u03b1\u03b9 \u03c3\u03c5\u03bd\u03b5\u03c7\u03ce\u03c2, \u03b1\u03c5\u03be\u03ac\u03bd\u03bf\u03bd\u03c4\u03b1\u03b9 \u03b7 \u03ba\u03b1\u03b8\u03c5\u03c3\u03c4\u03ad\u03c1\u03b7\u03c3\u03b7, \u03bf \u03ba\u03b1\u03c4\u03b1\u03ba\u03b5\u03c1\u03bc\u03b1\u03c4\u03b9\u03c3\u03bc\u03cc\u03c2 \u03bc\u03bd\u03ae\u03bc\u03b7\u03c2 \u03ba\u03b1\u03b9 \u03bf \u03ba\u03af\u03bd\u03b4\u03c5\u03bd\u03bf\u03c2 out-of-memory. \u03a4\u03bf MoonEP \u03bc\u03b5\u03c4\u03b1\u03c6\u03ad\u03c1\u03b5\u03b9 \u03b1\u03c5\u03c4\u03cc \u03c4\u03bf \u03c1\u03af\u03c3\u03ba\u03bf \u03c3\u03b5 \u03ad\u03bd\u03b1 \u03b1\u03c5\u03c3\u03c4\u03b7\u03c1\u03cc contract \u03b3\u03b9\u03b1 symmetric memory, prefetch slots \u03ba\u03b1\u03b9 gradient reduction.<\/p>\n<p>\u0393\u03b9\u03b1 \u03bc\u03b9\u03b1 \u03b5\u03c0\u03b9\u03c7\u03b5\u03af\u03c1\u03b7\u03c3\u03b7, \u03c4\u03bf \u03c3\u03c5\u03bc\u03c0\u03ad\u03c1\u03b1\u03c3\u03bc\u03b1 \u03b4\u03b5\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u00ab\u03b5\u03b3\u03ba\u03b1\u03c4\u03b1\u03c3\u03c4\u03ae\u03c3\u03c4\u03b5 \u03c4\u03bf MoonEP\u00bb. \u0395\u03af\u03bd\u03b1\u03b9 \u03bd\u03b1 \u03b1\u03c0\u03b1\u03b9\u03c4\u03b5\u03af \u03bc\u03b5\u03c4\u03c1\u03ae\u03c3\u03b5\u03b9\u03c2 \u03c3\u03c4\u03bf \u03b4\u03b9\u03ba\u03cc \u03c4\u03b7\u03c2 hardware, \u03bc\u03b5 \u03c4\u03bf \u03b4\u03b9\u03ba\u03cc \u03c4\u03b7\u03c2 routing profile, \u03c3\u03b1\u03c6\u03ae \u03cc\u03c1\u03b9\u03b1 \u03c3\u03c5\u03bc\u03b2\u03b1\u03c4\u03cc\u03c4\u03b7\u03c4\u03b1\u03c2 \u03ba\u03b1\u03b9 \u03b4\u03bf\u03ba\u03b9\u03bc\u03b1\u03c3\u03bc\u03ad\u03bd\u03bf fallback \u03c0\u03c1\u03b9\u03bd \u03b4\u03b5\u03c7\u03b8\u03b5\u03af \u03bc\u03b9\u03b1 \u03c5\u03c0\u03cc\u03c3\u03c7\u03b5\u03c3\u03b7 \u03c7\u03b1\u03bc\u03b7\u03bb\u03cc\u03c4\u03b5\u03c1\u03bf\u03c5 \u03ba\u03cc\u03c3\u03c4\u03bf\u03c5\u03c2 \u03ae \u03c5\u03c8\u03b7\u03bb\u03cc\u03c4\u03b5\u03c1\u03bf\u03c5 throughput.<\/p>\n<\/div>\n<div class=\"td-article-toc\">\n<div class=\"td-toc-title\">\u03a0\u03b5\u03c1\u03b9\u03b5\u03c7\u03cc\u03bc\u03b5\u03bd\u03b1<\/div>\n<ul>\n<li><a href=\"#moe-provlima-katanomis\">\u0393\u03b9\u03b1\u03c4\u03af \u03c4\u03b1 Mixture-of-Experts \u03b4\u03b7\u03bc\u03b9\u03bf\u03c5\u03c1\u03b3\u03bf\u03cd\u03bd \u03c0\u03c1\u03cc\u03b2\u03bb\u03b7\u03bc\u03b1 \u03ba\u03b1\u03c4\u03b1\u03bd\u03bf\u03bc\u03ae\u03c2<\/a><\/li>\n<li><a href=\"#moonep-stathero-fortio-s-k\">\u0397 \u03b2\u03b1\u03c3\u03b9\u03ba\u03ae \u03c5\u03c0\u03cc\u03c3\u03c7\u03b5\u03c3\u03b7 \u03c4\u03bf\u03c5 MoonEP: \u03c3\u03c4\u03b1\u03b8\u03b5\u03c1\u03cc \u03c6\u03bf\u03c1\u03c4\u03af\u03bf S \u00d7 K<\/a><\/li>\n<li><a href=\"#online-planning-zero-copy-static-shapes\">Online planning, zero copy \u03ba\u03b1\u03b9 \u03c3\u03c4\u03b1\u03c4\u03b9\u03ba\u03ac \u03c3\u03c7\u03ae\u03bc\u03b1\u03c4\u03b1<\/a><\/li>\n<li><a href=\"#memory-contract-techniki-omada\">\u03a4\u03bf memory contract \u03c0\u03bf\u03c5 \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03ba\u03b1\u03c4\u03b1\u03bb\u03ac\u03b2\u03b5\u03b9 \u03b7 \u03c4\u03b5\u03c7\u03bd\u03b9\u03ba\u03ae \u03bf\u03bc\u03ac\u03b4\u03b1<\/a><\/li>\n<li><a href=\"#gradients-ekpaidefsi\">\u03a4\u03b9 \u03c3\u03c5\u03bc\u03b2\u03b1\u03af\u03bd\u03b5\u03b9 \u03c3\u03c4\u03bf\u03c5\u03c2 gradients \u03ba\u03b1\u03c4\u03ac \u03c4\u03b7\u03bd \u03b5\u03ba\u03c0\u03b1\u03af\u03b4\u03b5\u03c5\u03c3\u03b7<\/a><\/li>\n<li><a href=\"#benchmarks-oria-apodeixeon\">\u03a4\u03b9 \u03b4\u03b5\u03af\u03c7\u03bd\u03bf\u03c5\u03bd \u2014 \u03ba\u03b1\u03b9 \u03c4\u03b9 \u03b4\u03b5\u03bd \u03b1\u03c0\u03bf\u03b4\u03b5\u03b9\u03ba\u03bd\u03cd\u03bf\u03c5\u03bd \u2014 \u03c4\u03b1 benchmarks<\/a><\/li>\n<li><a href=\"#epicheiriseis-ai-ypodomi\">\u0393\u03b9\u03b1\u03c4\u03af \u03b5\u03bd\u03b4\u03b9\u03b1\u03c6\u03ad\u03c1\u03b5\u03b9 \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03ae\u03c3\u03b5\u03b9\u03c2 \u03c0\u03bf\u03c5 \u03b1\u03b3\u03bf\u03c1\u03ac\u03b6\u03bf\u03c5\u03bd \u03ae \u03c7\u03c4\u03af\u03b6\u03bf\u03c5\u03bd AI<\/a><\/li>\n<li><a href=\"#plaisio-axiologisis-yiothetisis\">\u0388\u03bd\u03b1 \u03c1\u03b5\u03b1\u03bb\u03b9\u03c3\u03c4\u03b9\u03ba\u03cc \u03c0\u03bb\u03b1\u03af\u03c3\u03b9\u03bf \u03b1\u03be\u03b9\u03bf\u03bb\u03cc\u03b3\u03b7\u03c3\u03b7\u03c2 \u03c0\u03c1\u03b9\u03bd \u03b1\u03c0\u03cc \u03c4\u03b7\u03bd \u03c5\u03b9\u03bf\u03b8\u03ad\u03c4\u03b7\u03c3\u03b7<\/a><\/li>\n<\/ul>\n<\/div>\n<h2 id=\"moe-provlima-katanomis\">\u0393\u03b9\u03b1\u03c4\u03af \u03c4\u03b1 Mixture-of-Experts \u03b4\u03b7\u03bc\u03b9\u03bf\u03c5\u03c1\u03b3\u03bf\u03cd\u03bd \u03c0\u03c1\u03cc\u03b2\u03bb\u03b7\u03bc\u03b1 \u03ba\u03b1\u03c4\u03b1\u03bd\u03bf\u03bc\u03ae\u03c2<\/h2>\n<p>\u03a3\u03b5 \u03ad\u03bd\u03b1 Mixture-of-Experts \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03bf \u03b4\u03b5\u03bd \u03b5\u03bd\u03b5\u03c1\u03b3\u03bf\u03c0\u03bf\u03b9\u03bf\u03cd\u03bd\u03c4\u03b1\u03b9 \u03cc\u03bb\u03bf\u03b9 \u03bf\u03b9 experts \u03b3\u03b9\u03b1 \u03ba\u03ac\u03b8\u03b5 token. \u0388\u03bd\u03b1\u03c2 router \u03b5\u03c0\u03b9\u03bb\u03ad\u03b3\u03b5\u03b9 \u03c4\u03bf\u03c5\u03c2 \u03ba\u03bf\u03c1\u03c5\u03c6\u03b1\u03af\u03bf\u03c5\u03c2 K experts \u03ba\u03b1\u03b9 \u03c3\u03c4\u03ad\u03bb\u03bd\u03b5\u03b9 \u03c3\u03b5 \u03b1\u03c5\u03c4\u03bf\u03cd\u03c2 \u03c4\u03bf token. \u0397 \u03c4\u03b5\u03c7\u03bd\u03b9\u03ba\u03ae \u03b5\u03c0\u03b9\u03c4\u03c1\u03ad\u03c0\u03b5\u03b9 \u03c3\u03b5 \u03ad\u03bd\u03b1 \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03bf \u03bd\u03b1 \u03b4\u03b9\u03b1\u03b8\u03ad\u03c4\u03b5\u03b9 \u03c0\u03bf\u03bb\u03cd \u03bc\u03b5\u03b3\u03ac\u03bb\u03bf \u03c3\u03c5\u03bd\u03bf\u03bb\u03b9\u03ba\u03cc \u03b1\u03c1\u03b9\u03b8\u03bc\u03cc \u03c0\u03b1\u03c1\u03b1\u03bc\u03ad\u03c4\u03c1\u03c9\u03bd \u03c7\u03c9\u03c1\u03af\u03c2 \u03bd\u03b1 \u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03b5\u03af \u03cc\u03bb\u03b5\u03c2 \u03c4\u03b9\u03c2 \u03c0\u03b1\u03c1\u03b1\u03bc\u03ad\u03c4\u03c1\u03bf\u03c5\u03c2 \u03c3\u03b5 \u03ba\u03ac\u03b8\u03b5 \u03c5\u03c0\u03bf\u03bb\u03bf\u03b3\u03b9\u03c3\u03bc\u03cc. \u0397 \u03b1\u03c0\u03bf\u03b4\u03bf\u03c4\u03b9\u03ba\u03cc\u03c4\u03b7\u03c4\u03b1 \u03cc\u03bc\u03c9\u03c2 \u03c0\u03c1\u03bf\u03cb\u03c0\u03bf\u03b8\u03ad\u03c4\u03b5\u03b9 \u03cc\u03c4\u03b9 \u03b7 \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1 \u03ba\u03b1\u03c4\u03b1\u03bd\u03ad\u03bc\u03b5\u03c4\u03b1\u03b9 \u03b1\u03c1\u03ba\u03b5\u03c4\u03ac \u03bf\u03bc\u03bf\u03b9\u03cc\u03bc\u03bf\u03c1\u03c6\u03b1.<\/p>\n<p>\u03a3\u03c4\u03b7\u03bd \u03c0\u03c1\u03ac\u03be\u03b7, \u03bf\u03b9 \u03b1\u03c0\u03bf\u03c6\u03ac\u03c3\u03b5\u03b9\u03c2 \u03c4\u03bf\u03c5 router \u03b4\u03b5\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03c4\u03ad\u03bb\u03b5\u03b9\u03b1 \u03b9\u03c3\u03bf\u03c1\u03c1\u03bf\u03c0\u03b7\u03bc\u03ad\u03bd\u03b5\u03c2. \u039a\u03ac\u03c0\u03bf\u03b9\u03bf\u03b9 experts \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03bb\u03ac\u03b2\u03bf\u03c5\u03bd \u03c0\u03bf\u03bb\u03cd \u03c0\u03b5\u03c1\u03b9\u03c3\u03c3\u03cc\u03c4\u03b5\u03c1\u03b1 tokens. \u03a3\u03cd\u03bc\u03c6\u03c9\u03bd\u03b1 \u03bc\u03b5 \u03c4\u03b7\u03bd \u03c4\u03b5\u03c7\u03bd\u03b9\u03ba\u03ae \u03c0\u03b5\u03c1\u03b9\u03b3\u03c1\u03b1\u03c6\u03ae \u03c4\u03bf\u03c5 MoonEP, \u03b7 \u03ba\u03b1\u03b8\u03c5\u03c3\u03c4\u03ad\u03c1\u03b7\u03c3\u03b7 \u03bc\u03b9\u03b1\u03c2 \u03c3\u03c5\u03bb\u03bb\u03bf\u03b3\u03b9\u03ba\u03ae\u03c2 \u03bb\u03b5\u03b9\u03c4\u03bf\u03c5\u03c1\u03b3\u03af\u03b1\u03c2 \u03ba\u03b1\u03b8\u03bf\u03c1\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc \u03c4\u03bf\u03bd \u03c0\u03b9\u03bf \u03b1\u03c1\u03b3\u03cc \u03c3\u03c5\u03bc\u03bc\u03b5\u03c4\u03ad\u03c7\u03bf\u03bd\u03c4\u03b1. \u0386\u03c1\u03b1, \u03b1\u03ba\u03cc\u03bc\u03b7 \u03ba\u03b1\u03b9 \u03b1\u03bd \u03bf\u03b9 \u03c0\u03b5\u03c1\u03b9\u03c3\u03c3\u03cc\u03c4\u03b5\u03c1\u03b5\u03c2 GPU \u03c4\u03b5\u03bb\u03b5\u03b9\u03ce\u03c3\u03bf\u03c5\u03bd \u03b3\u03c1\u03ae\u03b3\u03bf\u03c1\u03b1, \u03c4\u03bf \u03b2\u03ae\u03bc\u03b1 \u03b5\u03ba\u03c0\u03b1\u03af\u03b4\u03b5\u03c5\u03c3\u03b7\u03c2 \u03c0\u03b5\u03c1\u03b9\u03bc\u03ad\u03bd\u03b5\u03b9 \u03c4\u03bf rank \u03bc\u03b5 \u03c4\u03bf \u03bc\u03b5\u03b3\u03b1\u03bb\u03cd\u03c4\u03b5\u03c1\u03bf \u03c6\u03bf\u03c1\u03c4\u03af\u03bf.<\/p>\n<p>\u0397 \u03b2\u03b9\u03b2\u03bb\u03b9\u03bf\u03b8\u03ae\u03ba\u03b7 \u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03b5\u03af \u03c4\u03bf\u03bd \u03b4\u03b5\u03af\u03ba\u03c4\u03b7 maxvio \u03b3\u03b9\u03b1 \u03bd\u03b1 \u03c0\u03b5\u03c1\u03b9\u03b3\u03c1\u03ac\u03c8\u03b5\u03b9 \u03c4\u03b7\u03bd \u03b1\u03c0\u03cc\u03ba\u03bb\u03b9\u03c3\u03b7 \u03b1\u03c0\u03cc \u03c4\u03b7\u03bd \u03b9\u03b4\u03b1\u03bd\u03b9\u03ba\u03ae \u03ba\u03b1\u03c4\u03b1\u03bd\u03bf\u03bc\u03ae. \u03a4\u03b9\u03bc\u03ae \u03bc\u03b7\u03b4\u03ad\u03bd \u03c3\u03b7\u03bc\u03b1\u03af\u03bd\u03b5\u03b9 \u03c4\u03ad\u03bb\u03b5\u03b9\u03b1 \u03b9\u03c3\u03bf\u03c1\u03c1\u03bf\u03c0\u03af\u03b1. \u039f \u03b4\u03b5\u03af\u03ba\u03c4\u03b7\u03c2 \u03b4\u03b5\u03bd \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03b5\u03af \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03b7\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc KPI, \u03b1\u03bb\u03bb\u03ac \u03b2\u03bf\u03b7\u03b8\u03ac \u03c4\u03b7\u03bd \u03c4\u03b5\u03c7\u03bd\u03b9\u03ba\u03ae \u03bf\u03bc\u03ac\u03b4\u03b1 \u03bd\u03b1 \u03c3\u03c5\u03bd\u03b4\u03ad\u03c3\u03b5\u03b9 \u03c4\u03b7 \u03c3\u03c5\u03bc\u03c0\u03b5\u03c1\u03b9\u03c6\u03bf\u03c1\u03ac \u03c4\u03bf\u03c5 router \u03bc\u03b5 \u03c4\u03b7\u03bd \u03b5\u03c0\u03b9\u03ba\u03bf\u03b9\u03bd\u03c9\u03bd\u03af\u03b1 \u03ba\u03b1\u03b9 \u03c4\u03b7 \u03b4\u03b9\u03ac\u03c1\u03ba\u03b5\u03b9\u03b1 \u03ba\u03ac\u03b8\u03b5 iteration. \u0397 \u03af\u03b4\u03b9\u03b1 \u03c0\u03b5\u03b9\u03b8\u03b1\u03c1\u03c7\u03af\u03b1 \u2014\u03bd\u03b1 \u03b3\u03bd\u03c9\u03c1\u03af\u03b6\u03bf\u03c5\u03bc\u03b5 \u03b1\u03ba\u03c1\u03b9\u03b2\u03ce\u03c2 \u03c4\u03b9 \u03bc\u03b5\u03c4\u03c1\u03ac \u03ad\u03bd\u03b1 benchmark\u2014 \u03b5\u03af\u03bd\u03b1\u03b9 \u03ba\u03c1\u03af\u03c3\u03b9\u03bc\u03b7 \u03ba\u03b1\u03b9 \u03c3\u03c4\u03b7\u03bd <a href=\"https:\/\/twodots.gr\/mag-neo-benchmark-ai-agents-sto-web\/\">\u03b1\u03be\u03b9\u03bf\u03bb\u03cc\u03b3\u03b7\u03c3\u03b7 AI agents \u03c3\u03c4\u03bf web<\/a>.<\/p>\n<div class=\"td-article-note\">\n<p><strong>\u03a4\u03b9 \u03bd\u03b1 \u03ba\u03c1\u03b1\u03c4\u03ae\u03c3\u03b5\u03c4\u03b5:<\/strong> \u03c4\u03bf bottleneck \u03b4\u03b5\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03bf \u03bc\u03ad\u03c3\u03bf\u03c2 \u03c6\u03cc\u03c1\u03c4\u03bf\u03c2 \u03c4\u03c9\u03bd experts \u03b1\u03bb\u03bb\u03ac \u03bf \u03c0\u03b9\u03bf \u00ab\u03b8\u03b5\u03c1\u03bc\u03cc\u03c2\u00bb rank \u03c0\u03bf\u03c5 \u03ba\u03b1\u03b8\u03c5\u03c3\u03c4\u03b5\u03c1\u03b5\u03af \u03c4\u03b7 \u03c3\u03c5\u03bb\u03bb\u03bf\u03b3\u03b9\u03ba\u03ae \u03bb\u03b5\u03b9\u03c4\u03bf\u03c5\u03c1\u03b3\u03af\u03b1. \u03a4\u03bf MoonEP \u03b4\u03b5\u03bd \u03b1\u03bb\u03bb\u03ac\u03b6\u03b5\u03b9 \u03c4\u03bf routing \u03c4\u03bf\u03c5 \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03bf\u03c5\u00b7 \u03c3\u03c7\u03b5\u03b4\u03b9\u03ac\u03b6\u03b5\u03b9 \u03c0\u03c1\u03bf\u03c3\u03c9\u03c1\u03b9\u03bd\u03ac \u03b1\u03bd\u03c4\u03af\u03b3\u03c1\u03b1\u03c6\u03b1 experts \u03ce\u03c3\u03c4\u03b5 \u03bf \u03c5\u03c0\u03bf\u03bb\u03bf\u03b3\u03b9\u03c3\u03bc\u03cc\u03c2 \u03bd\u03b1 \u03bc\u03bf\u03b9\u03c1\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03c0\u03c1\u03bf\u03b2\u03bb\u03ad\u03c8\u03b9\u03bc\u03b1.<\/p>\n<\/div>\n<h2 id=\"moonep-stathero-fortio-s-k\">\u0397 \u03b2\u03b1\u03c3\u03b9\u03ba\u03ae \u03c5\u03c0\u03cc\u03c3\u03c7\u03b5\u03c3\u03b7 \u03c4\u03bf\u03c5 MoonEP: \u03c3\u03c4\u03b1\u03b8\u03b5\u03c1\u03cc \u03c6\u03bf\u03c1\u03c4\u03af\u03bf S \u00d7 K<\/h2>\n<p>\u0397 \u03ba\u03b5\u03bd\u03c4\u03c1\u03b9\u03ba\u03ae \u03b9\u03b4\u03ad\u03b1 \u03c4\u03bf\u03c5 MoonEP \u03b5\u03af\u03bd\u03b1\u03b9 \u03ad\u03bd\u03b1 \u03c3\u03b1\u03c6\u03ad\u03c2 invariant: \u03ba\u03ac\u03b8\u03b5 rank \u03bb\u03b1\u03bc\u03b2\u03ac\u03bd\u03b5\u03b9 \u03b1\u03ba\u03c1\u03b9\u03b2\u03ce\u03c2 S \u00d7 K tokens, \u03b1\u03bd\u03b5\u03be\u03ac\u03c1\u03c4\u03b7\u03c4\u03b1 \u03b1\u03c0\u03cc \u03c4\u03bf \u03c0\u03cc\u03c3\u03bf \u03ac\u03bd\u03b9\u03c3\u03b7 \u03b5\u03af\u03bd\u03b1\u03b9 \u03b7 \u03b1\u03c1\u03c7\u03b9\u03ba\u03ae \u03b4\u03c1\u03bf\u03bc\u03bf\u03bb\u03cc\u03b3\u03b7\u03c3\u03b7. \u03a4\u03bf S \u03b5\u03af\u03bd\u03b1\u03b9 \u03bf \u03b1\u03c1\u03b9\u03b8\u03bc\u03cc\u03c2 \u03b5\u03b9\u03c3\u03b5\u03c1\u03c7\u03cc\u03bc\u03b5\u03bd\u03c9\u03bd tokens \u03b1\u03bd\u03ac rank \u03ba\u03b1\u03b9 \u03c4\u03bf K \u03bf \u03b1\u03c1\u03b9\u03b8\u03bc\u03cc\u03c2 experts \u03c0\u03bf\u03c5 \u03b5\u03c0\u03b9\u03bb\u03ad\u03b3\u03bf\u03bd\u03c4\u03b1\u03b9 \u03b3\u03b9\u03b1 \u03ba\u03ac\u03b8\u03b5 token. \u0397 \u03c5\u03c0\u03cc\u03c3\u03c7\u03b5\u03c3\u03b7 \u03b1\u03c5\u03c4\u03ae \u03c0\u03b1\u03c1\u03bf\u03c5\u03c3\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc \u03c4\u03bf\u03c5\u03c2 \u03b4\u03b7\u03bc\u03b9\u03bf\u03c5\u03c1\u03b3\u03bf\u03cd\u03c2 \u03c4\u03b7\u03c2 \u03b2\u03b9\u03b2\u03bb\u03b9\u03bf\u03b8\u03ae\u03ba\u03b7\u03c2 \u03ba\u03b1\u03b9 \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03b1\u03be\u03b9\u03bf\u03bb\u03bf\u03b3\u03b5\u03af\u03c4\u03b1\u03b9 \u03c3\u03c4\u03bf \u03c0\u03bb\u03b1\u03af\u03c3\u03b9\u03bf \u03c4\u03c9\u03bd \u03c5\u03c0\u03bf\u03c3\u03c4\u03b7\u03c1\u03b9\u03b6\u03cc\u03bc\u03b5\u03bd\u03c9\u03bd \u03c1\u03c5\u03b8\u03bc\u03af\u03c3\u03b5\u03c9\u03bd \u03ba\u03b1\u03b9 \u03c4\u03bf\u03c5 \u03b4\u03b9\u03ba\u03bf\u03cd \u03c4\u03bf\u03c5\u03c2 \u03ba\u03ce\u03b4\u03b9\u03ba\u03b1.<\/p>\n<p>\u0393\u03b9\u03b1 \u03bd\u03b1 \u03c0\u03b5\u03c4\u03cd\u03c7\u03b5\u03b9 \u03c4\u03b7\u03bd \u03b9\u03c3\u03bf\u03c1\u03c1\u03bf\u03c0\u03af\u03b1, \u03c4\u03bf \u03c3\u03cd\u03c3\u03c4\u03b7\u03bc\u03b1 \u03b4\u03b7\u03bc\u03b9\u03bf\u03c5\u03c1\u03b3\u03b5\u03af \u03b4\u03c5\u03bd\u03b1\u03bc\u03b9\u03ba\u03ac \u03ad\u03bd\u03b1\u03bd \u03bc\u03b9\u03ba\u03c1\u03cc \u03b1\u03c1\u03b9\u03b8\u03bc\u03cc redundant experts. \u039c\u03b5 \u03b1\u03c0\u03bb\u03ac \u03bb\u03cc\u03b3\u03b9\u03b1, \u03b1\u03bd\u03c4\u03b9\u03b3\u03c1\u03ac\u03c6\u03b5\u03b9 \u03c0\u03c1\u03bf\u03c3\u03c9\u03c1\u03b9\u03bd\u03ac \u03c4\u03bf\u03c5\u03c2 experts \u03c0\u03bf\u03c5 \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 \u03c3\u03b5 \u03ac\u03bb\u03bb\u03b1 ranks, \u03b1\u03bd\u03c4\u03af \u03bd\u03b1 \u03b1\u03c6\u03ae\u03c3\u03b5\u03b9 \u03cc\u03bb\u03b7 \u03c4\u03b7\u03bd \u03b1\u03c5\u03be\u03b7\u03bc\u03ad\u03bd\u03b7 \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1 \u03c3\u03c4\u03bf \u03b1\u03c1\u03c7\u03b9\u03ba\u03cc rank. \u039f \u03c3\u03c7\u03b5\u03b4\u03b9\u03b1\u03c3\u03bc\u03cc\u03c2 \u03b3\u03af\u03bd\u03b5\u03c4\u03b1\u03b9 online, \u03bc\u03b5 \u03b2\u03ac\u03c3\u03b7 \u03c4\u03b9\u03c2 \u03c4\u03c1\u03ad\u03c7\u03bf\u03c5\u03c3\u03b5\u03c2 \u03b5\u03be\u03cc\u03b4\u03bf\u03c5\u03c2 \u03c4\u03bf\u03c5 router, \u03ba\u03b1\u03b9 \u03c4\u03b1 \u03b1\u03bd\u03c4\u03af\u03b3\u03c1\u03b1\u03c6\u03b1 \u03c4\u03c9\u03bd \u03b2\u03b1\u03c1\u03ce\u03bd \u03c0\u03c1\u03bf\u03c6\u03bf\u03c1\u03c4\u03ce\u03bd\u03bf\u03bd\u03c4\u03b1\u03b9 \u03c0\u03c1\u03b9\u03bd \u03b1\u03c0\u03cc \u03c4\u03bf\u03bd \u03c5\u03c0\u03bf\u03bb\u03bf\u03b3\u03b9\u03c3\u03bc\u03cc \u03c4\u03c9\u03bd experts.<\/p>\n<p>\u039a\u03b1\u03c4\u03ac \u03c4\u03bf backward pass, \u03bf\u03b9 gradients \u03b1\u03c5\u03c4\u03ce\u03bd \u03c4\u03c9\u03bd \u03c0\u03c1\u03bf\u03c3\u03c9\u03c1\u03b9\u03bd\u03ce\u03bd \u03b1\u03bd\u03c4\u03b9\u03b3\u03c1\u03ac\u03c6\u03c9\u03bd \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03b5\u03c0\u03b9\u03c3\u03c4\u03c1\u03ad\u03c8\u03bf\u03c5\u03bd \u03c3\u03c4\u03bf\u03c5\u03c2 home ranks. \u0395\u03c0\u03bf\u03bc\u03ad\u03bd\u03c9\u03c2, \u03b7 \u03c4\u03b5\u03c7\u03bd\u03b9\u03ba\u03ae \u03b4\u03b5\u03bd \u03b5\u03be\u03b1\u03c6\u03b1\u03bd\u03af\u03b6\u03b5\u03b9 \u03c4\u03b7\u03bd \u03c0\u03bf\u03bb\u03c5\u03c0\u03bb\u03bf\u03ba\u03cc\u03c4\u03b7\u03c4\u03b1. \u03a4\u03b7 \u03bc\u03b5\u03c4\u03b1\u03c6\u03ad\u03c1\u03b5\u03b9 \u03c3\u03b5 \u03ad\u03bd\u03b1\u03bd \u03c0\u03c1\u03bf\u03b3\u03c1\u03b1\u03bc\u03bc\u03b1\u03c4\u03b9\u03c3\u03bc\u03ad\u03bd\u03bf \u03bc\u03b7\u03c7\u03b1\u03bd\u03b9\u03c3\u03bc\u03cc \u03b1\u03bd\u03c4\u03b9\u03b3\u03c1\u03b1\u03c6\u03ae\u03c2, prefetch \u03ba\u03b1\u03b9 gradient reduction, \u03bc\u03b5 \u03c3\u03c4\u03cc\u03c7\u03bf \u03bd\u03b1 \u03b1\u03c0\u03bf\u03c6\u03cd\u03b3\u03b5\u03b9 \u03c4\u03b7\u03bd \u03ba\u03b1\u03b8\u03c5\u03c3\u03c4\u03ad\u03c1\u03b7\u03c3\u03b7 \u03c0\u03bf\u03c5 \u03b4\u03b7\u03bc\u03b9\u03bf\u03c5\u03c1\u03b3\u03b5\u03af \u03bf \u03c0\u03b9\u03bf \u00ab\u03b8\u03b5\u03c1\u03bc\u03cc\u03c2\u00bb expert.<\/p>\n<div class=\"td-comparison\">\n<p class=\"td-comparison-title\">\u03a4\u03ad\u03c3\u03c3\u03b5\u03c1\u03b9\u03c2 \u03cc\u03c8\u03b5\u03b9\u03c2 \u03c4\u03b7\u03c2 \u03b5\u03be\u03b9\u03c3\u03bf\u03c1\u03c1\u03cc\u03c0\u03b7\u03c3\u03b7\u03c2 experts<\/p>\n<div class=\"td-comparison-cards td-comparison-cards--horizontal\">\n<div class=\"td-comparison-grid td-comparison-grid--two\">\n<div class=\"td-platform-card\">\n<h3>\u03a7\u03c9\u03c1\u03af\u03c2 \u03b1\u03bd\u03b1\u03ba\u03b1\u03c4\u03b1\u03bd\u03bf\u03bc\u03ae<\/h3>\n<p>\u039f expert \u03bc\u03b5 \u03c4\u03b7 \u03bc\u03b5\u03b3\u03b1\u03bb\u03cd\u03c4\u03b5\u03c1\u03b7 \u03b6\u03ae\u03c4\u03b7\u03c3\u03b7 \u03c3\u03c5\u03b3\u03ba\u03b5\u03bd\u03c4\u03c1\u03ce\u03bd\u03b5\u03b9 \u03c0\u03b5\u03c1\u03b9\u03c3\u03c3\u03cc\u03c4\u03b5\u03c1\u03b1 tokens \u03c3\u03c4\u03bf\u03bd home rank. \u039f\u03b9 \u03c5\u03c0\u03cc\u03bb\u03bf\u03b9\u03c0\u03bf\u03b9 ranks \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03bf\u03bb\u03bf\u03ba\u03bb\u03b7\u03c1\u03ce\u03c3\u03bf\u03c5\u03bd \u03bd\u03c9\u03c1\u03af\u03c4\u03b5\u03c1\u03b1, \u03b1\u03bb\u03bb\u03ac \u03c4\u03bf iteration \u03c0\u03b5\u03c1\u03b9\u03bc\u03ad\u03bd\u03b5\u03b9 \u03c4\u03bf \u03c0\u03b9\u03bf \u03b1\u03c1\u03b3\u03cc \u03c3\u03b7\u03bc\u03b5\u03af\u03bf.<\/p>\n<div class=\"td-badge-row\"><span class=\"td-badge\">\u039c\u03b5\u03c4\u03b1\u03b2\u03bb\u03b7\u03c4\u03cc \u03c6\u03bf\u03c1\u03c4\u03af\u03bf<\/span><span class=\"td-badge\">\u0394\u03c5\u03bd\u03b1\u03bc\u03b9\u03ba\u03ac shapes<\/span><\/div>\n<\/div>\n<div class=\"td-platform-card td-platform-card--navy\">\n<h3>\u039c\u03b5 \u03c4\u03bf\u03bd \u03c3\u03c7\u03b5\u03b4\u03b9\u03b1\u03c3\u03bc\u03cc \u03c4\u03bf\u03c5 MoonEP<\/h3>\n<p>\u039f GPU planner \u03b5\u03c0\u03b9\u03bb\u03ad\u03b3\u03b5\u03b9 \u03c0\u03bf\u03b9\u03bf\u03b9 experts \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 \u03c0\u03c1\u03bf\u03c3\u03c9\u03c1\u03b9\u03bd\u03ac \u03b1\u03bd\u03c4\u03af\u03b3\u03c1\u03b1\u03c6\u03b1, \u03c0\u03c1\u03bf\u03c6\u03bf\u03c1\u03c4\u03ce\u03bd\u03b5\u03b9 \u03c4\u03b1 \u03b2\u03ac\u03c1\u03b7 \u03c3\u03b5 \u03ac\u03bb\u03bb\u03bf\u03c5\u03c2 ranks \u03ba\u03b1\u03b9 \u03b5\u03c0\u03b9\u03c3\u03c4\u03c1\u03ad\u03c6\u03b5\u03b9 \u03c4\u03bf\u03c5\u03c2 gradients \u03c3\u03c4\u03bf\u03c5\u03c2 home ranks \u03ba\u03b1\u03c4\u03ac \u03c4\u03bf backward pass.<\/p>\n<div class=\"td-badge-row\"><span class=\"td-badge\">S \u00d7 K \u03b1\u03bd\u03ac rank<\/span><span class=\"td-badge\">Prefetch \u03ba\u03b1\u03b9 reduce<\/span><\/div>\n<\/div>\n<div class=\"td-platform-card\">\n<h3>\u039a\u03cc\u03c3\u03c4\u03bf\u03c2 \u03c0\u03bf\u03c5 \u03bc\u03b5\u03c4\u03b1\u03c6\u03ad\u03c1\u03b5\u03c4\u03b1\u03b9<\/h3>\n<p>\u0397 \u03b5\u03be\u03b9\u03c3\u03bf\u03c1\u03c1\u03cc\u03c0\u03b7\u03c3\u03b7 \u03b1\u03c0\u03b1\u03b9\u03c4\u03b5\u03af \u03c3\u03c5\u03bd\u03b5\u03c7\u03cc\u03bc\u03b5\u03bd\u03b1 symmetric-memory tensors, prefetch slots B \u03ba\u03b1\u03b9 \u03be\u03b5\u03c7\u03c9\u03c1\u03b9\u03c3\u03c4\u03ac reduce buffers \u03b3\u03b9\u03b1 \u03c4\u03bf\u03c5\u03c2 gradients \u03c4\u03c9\u03bd duplicated experts.<\/p>\n<div class=\"td-badge-row\"><span class=\"td-badge\">Memory contract<\/span><span class=\"td-badge\">NVLink reads<\/span><\/div>\n<\/div>\n<div class=\"td-platform-card td-platform-card--navy\">\n<h3>\u0391\u03c0\u03cc\u03b4\u03b5\u03b9\u03be\u03b7 \u03c0\u03bf\u03c5 \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9<\/h3>\n<p>\u0397 \u03b1\u03be\u03b9\u03bf\u03bb\u03cc\u03b3\u03b7\u03c3\u03b7 \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03c0\u03b5\u03c1\u03b9\u03bb\u03b1\u03bc\u03b2\u03ac\u03bd\u03b5\u03b9 planning, prefetch, dispatch, combine, group GEMM, gradient reduction, peak \u03bc\u03bd\u03ae\u03bc\u03b7 \u03ba\u03b1\u03b9 OOM \u03c3\u03c4\u03bf \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc routing profile.<\/p>\n<div class=\"td-badge-row\"><span class=\"td-badge\">Critical path<\/span><span class=\"td-badge\">\u0394\u03b9\u03ba\u03cc \u03c3\u03b1\u03c2 workload<\/span><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<h2 id=\"online-planning-zero-copy-static-shapes\">Online planning, zero copy \u03ba\u03b1\u03b9 \u03c3\u03c4\u03b1\u03c4\u03b9\u03ba\u03ac \u03c3\u03c7\u03ae\u03bc\u03b1\u03c4\u03b1<\/h2>\n<p>\u0397 \u03b1\u03c1\u03c7\u03b9\u03c4\u03b5\u03ba\u03c4\u03bf\u03bd\u03b9\u03ba\u03ae \u03c3\u03c4\u03b7\u03c1\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 \u03c3\u03b5 \u03c4\u03c1\u03b5\u03b9\u03c2 \u03b9\u03b4\u03b9\u03cc\u03c4\u03b7\u03c4\u03b5\u03c2. \u0397 \u03c0\u03c1\u03ce\u03c4\u03b7 \u03b5\u03af\u03bd\u03b1\u03b9 \u03b7 \u03b9\u03c3\u03bf\u03c1\u03c1\u03bf\u03c0\u03af\u03b1 \u03bc\u03ad\u03c3\u03c9 redundant experts. \u0397 \u03b4\u03b5\u03cd\u03c4\u03b5\u03c1\u03b7 \u03b5\u03af\u03bd\u03b1\u03b9 \u03bf online planner \u03c0\u03bf\u03c5 \u03b5\u03ba\u03c4\u03b5\u03bb\u03b5\u03af\u03c4\u03b1\u03b9 \u03c3\u03c4\u03b7 GPU. \u03a4\u03bf \u03b1\u03c0\u03bf\u03b8\u03b5\u03c4\u03ae\u03c1\u03b9\u03bf \u03b1\u03bd\u03b1\u03c6\u03ad\u03c1\u03b5\u03b9 \u03cc\u03c4\u03b9 \u03bf planner \u03ad\u03c7\u03b5\u03b9 \u03b1\u03bc\u03b5\u03bb\u03b7\u03c4\u03ad\u03bf overhead \u03ba\u03b1\u03b9 \u03c5\u03bb\u03bf\u03c0\u03bf\u03b9\u03b5\u03af\u03c4\u03b1\u03b9 \u03bc\u03b5 CUTLASS CuTe DSL. \u03a0\u03c1\u03cc\u03ba\u03b5\u03b9\u03c4\u03b1\u03b9 \u03b3\u03b9\u03b1 \u03b4\u03ae\u03bb\u03c9\u03c3\u03b7 \u03c4\u03c9\u03bd \u03b4\u03b7\u03bc\u03b9\u03bf\u03c5\u03c1\u03b3\u03ce\u03bd \u03ba\u03b1\u03b9 \u03cc\u03c7\u03b9 \u03b3\u03b9\u03b1 \u03b1\u03bd\u03b5\u03be\u03ac\u03c1\u03c4\u03b7\u03c4\u03b7 \u03bc\u03ad\u03c4\u03c1\u03b7\u03c3\u03b7 \u03c3\u03c4\u03bf \u03c0\u03bb\u03b1\u03af\u03c3\u03b9\u03bf \u03b1\u03c5\u03c4\u03bf\u03cd \u03c4\u03bf\u03c5 \u03ac\u03c1\u03b8\u03c1\u03bf\u03c5.<\/p>\n<p>\u0397 \u03c4\u03c1\u03af\u03c4\u03b7 \u03b9\u03b4\u03b9\u03cc\u03c4\u03b7\u03c4\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 \u03bf \u03c3\u03c5\u03bd\u03b4\u03c5\u03b1\u03c3\u03bc\u03cc\u03c2 zero-copy \u03b5\u03c0\u03b9\u03ba\u03bf\u03b9\u03bd\u03c9\u03bd\u03af\u03b1\u03c2 \u03ba\u03b1\u03b9 \u03c3\u03c4\u03b1\u03c4\u03b9\u03ba\u03ce\u03bd shapes. \u03a4\u03b1 tokens \u03b3\u03c1\u03ac\u03c6\u03bf\u03bd\u03c4\u03b1\u03b9 \u03b1\u03c0\u03b5\u03c5\u03b8\u03b5\u03af\u03b1\u03c2 \u03c3\u03c4\u03b9\u03c2 \u03c4\u03b5\u03bb\u03b9\u03ba\u03ad\u03c2 \u03b8\u03ad\u03c3\u03b5\u03b9\u03c2 \u03c4\u03bf\u03c5\u03c2, \u03bf\u03bc\u03b1\u03b4\u03bf\u03c0\u03bf\u03b9\u03b7\u03bc\u03ad\u03bd\u03b1 \u03b1\u03bd\u03ac expert \u03c3\u03c4\u03b1 \u03b1\u03c0\u03bf\u03bc\u03b1\u03ba\u03c1\u03c5\u03c3\u03bc\u03ad\u03bd\u03b1 ranks, \u03ba\u03b1\u03b9 \u03b7 \u03b5\u03c6\u03b1\u03c1\u03bc\u03bf\u03b3\u03ae \u03bb\u03b1\u03bc\u03b2\u03ac\u03bd\u03b5\u03b9 views \u03c4\u03bf\u03c5 communication buffer. \u0388\u03c4\u03c3\u03b9 \u03b1\u03c0\u03bf\u03c6\u03b5\u03cd\u03b3\u03b5\u03c4\u03b1\u03b9 \u03ad\u03bd\u03b1 \u03b5\u03c0\u03b9\u03c0\u03bb\u03ad\u03bf\u03bd \u03b1\u03bd\u03c4\u03af\u03b3\u03c1\u03b1\u03c6\u03bf \u03b1\u03c0\u03cc communication buffer \u03c3\u03b5 user buffer.<\/p>\n<p>\u03a4\u03b1 \u03c3\u03c4\u03b1\u03c4\u03b9\u03ba\u03ac shapes \u03b5\u03af\u03bd\u03b1\u03b9 \u03b5\u03be\u03af\u03c3\u03bf\u03c5 \u03c3\u03b7\u03bc\u03b1\u03bd\u03c4\u03b9\u03ba\u03ac. \u038c\u03c4\u03b1\u03bd \u03c4\u03bf \u03bc\u03ad\u03b3\u03b5\u03b8\u03bf\u03c2 \u03c4\u03c9\u03bd activations \u03b1\u03bb\u03bb\u03ac\u03b6\u03b5\u03b9 \u03c3\u03b5 \u03ba\u03ac\u03b8\u03b5 \u03b2\u03ae\u03bc\u03b1, \u03b7 GPU memory \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03ba\u03b1\u03c4\u03b1\u03ba\u03b5\u03c1\u03bc\u03b1\u03c4\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 \u03ba\u03b1\u03b9 \u03b7 host \u03c0\u03bb\u03b5\u03c5\u03c1\u03ac \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03c3\u03c5\u03b3\u03c7\u03c1\u03bf\u03bd\u03b9\u03c3\u03bc\u03cc \u03b3\u03b9\u03b1 \u03bd\u03b1 \u03bc\u03ac\u03b8\u03b5\u03b9 \u03c4\u03b1 \u03bd\u03ad\u03b1 \u03bc\u03b5\u03b3\u03ad\u03b8\u03b7. \u03a4\u03bf MoonEP \u03b5\u03c0\u03b9\u03b4\u03b9\u03ce\u03ba\u03b5\u03b9 \u03c3\u03c4\u03b1\u03b8\u03b5\u03c1\u03cc buffer S \u00d7 K, \u03ce\u03c3\u03c4\u03b5 \u03c4\u03b1 \u03c3\u03c7\u03ae\u03bc\u03b1\u03c4\u03b1 \u03bd\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 \u03b3\u03bd\u03c9\u03c3\u03c4\u03ac \u03b5\u03ba \u03c4\u03c9\u03bd \u03c0\u03c1\u03bf\u03c4\u03ad\u03c1\u03c9\u03bd \u03ba\u03b1\u03b9 \u03bd\u03b1 \u03b1\u03c6\u03b1\u03b9\u03c1\u03b5\u03af\u03c4\u03b1\u03b9 \u03bf per-layer host synchronization \u03c0\u03bf\u03c5 \u03c3\u03c5\u03bd\u03b4\u03ad\u03b5\u03c4\u03b1\u03b9 \u03bc\u03b5 \u03b4\u03c5\u03bd\u03b1\u03bc\u03b9\u03ba\u03ac \u03bc\u03b5\u03b3\u03ad\u03b8\u03b7.<\/p>\n<h2 id=\"memory-contract-techniki-omada\">\u03a4\u03bf memory contract \u03c0\u03bf\u03c5 \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03ba\u03b1\u03c4\u03b1\u03bb\u03ac\u03b2\u03b5\u03b9 \u03b7 \u03c4\u03b5\u03c7\u03bd\u03b9\u03ba\u03ae \u03bf\u03bc\u03ac\u03b4\u03b1<\/h2>\n<p>\u03a4\u03bf MoonEP \u03b4\u03b5\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03ad\u03bd\u03b1 \u03b3\u03b5\u03bd\u03b9\u03ba\u03cc optimization \u03c0\u03bf\u03c5 \u03b5\u03bd\u03b5\u03c1\u03b3\u03bf\u03c0\u03bf\u03b9\u03b5\u03af\u03c4\u03b1\u03b9 \u03c7\u03c9\u03c1\u03af\u03c2 \u03b1\u03bb\u03bb\u03b1\u03b3\u03ad\u03c2. \u0391\u03c0\u03b1\u03b9\u03c4\u03b5\u03af \u03c3\u03c5\u03b3\u03ba\u03b5\u03ba\u03c1\u03b9\u03bc\u03ad\u03bd\u03bf \u03c3\u03c5\u03bc\u03b2\u03cc\u03bb\u03b1\u03b9\u03bf \u03bc\u03bd\u03ae\u03bc\u03b7\u03c2: \u03ad\u03bd\u03b1 \u03c3\u03c5\u03bd\u03b5\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf symmetric-memory tensor \u03b2\u03b1\u03c1\u03ce\u03bd \u03b3\u03b9\u03b1 \u03ba\u03ac\u03b8\u03b5 expert projection \u03ba\u03b1\u03b9 \u03ad\u03bd\u03b1 cu_seqlens \u03c0\u03bf\u03c5 \u03c0\u03b1\u03c1\u03ac\u03b3\u03b5\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc \u03c4\u03bf\u03bd planner. \u03a4\u03bf group GEMM \u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03b5\u03af tensor \u03c3\u03c7\u03ae\u03bc\u03b1\u03c4\u03bf\u03c2 [E+B, H, H&#8217;], \u03cc\u03c0\u03bf\u03c5 E \u03b5\u03af\u03bd\u03b1\u03b9 \u03c4\u03bf \u03c3\u03cd\u03bd\u03bf\u03bb\u03bf \u03c4\u03c9\u03bd routed experts, B \u03bf\u03b9 \u03b8\u03ad\u03c3\u03b5\u03b9\u03c2 prefetch \u03b1\u03bd\u03ac rank, H \u03c4\u03bf hidden size \u03ba\u03b1\u03b9 H&#8217; \u03b7 \u03b5\u03bd\u03b4\u03b9\u03ac\u03bc\u03b5\u03c3\u03b7 \u03b4\u03b9\u03ac\u03c3\u03c4\u03b1\u03c3\u03b7 \u03c4\u03bf\u03c5 expert FFN.<\/p>\n<p>\u039f\u03b9 \u03b3\u03c1\u03b1\u03bc\u03bc\u03ad\u03c2 \u03b1\u03c0\u03cc 0 \u03ad\u03c9\u03c2 E\u22121 \u03c6\u03b9\u03bb\u03bf\u03be\u03b5\u03bd\u03bf\u03cd\u03bd \u03c4\u03bf\u03c5\u03c2 experts \u03c4\u03c9\u03bd ranks. \u039f\u03b9 \u03c0\u03c1\u03cc\u03c3\u03b8\u03b5\u03c4\u03b5\u03c2 \u03b3\u03c1\u03b1\u03bc\u03bc\u03ad\u03c2 \u03b1\u03c0\u03cc E \u03ad\u03c9\u03c2 E+B \u03bb\u03b5\u03b9\u03c4\u03bf\u03c5\u03c1\u03b3\u03bf\u03cd\u03bd \u03c9\u03c2 \u03c4\u03bf\u03c0\u03b9\u03ba\u03ad\u03c2 \u03b8\u03ad\u03c3\u03b5\u03b9\u03c2 prefetch. \u0397 \u03c3\u03c5\u03bd\u03ad\u03c7\u03b5\u03b9\u03b1 \u03c4\u03b7\u03c2 \u03bc\u03bd\u03ae\u03bc\u03b7\u03c2 \u03b5\u03af\u03bd\u03b1\u03b9 \u03b1\u03c0\u03b1\u03af\u03c4\u03b7\u03c3\u03b7, \u03b5\u03c0\u03b5\u03b9\u03b4\u03ae \u03bf group GEMM \u03b5\u03bd\u03c4\u03bf\u03c0\u03af\u03b6\u03b5\u03b9 \u03c4\u03bf\u03c5\u03c2 experts \u03bc\u03ad\u03c3\u03c9 \u03c4\u03bf\u03c5 row index. \u0391\u03c5\u03c4\u03cc \u03c3\u03b7\u03bc\u03b1\u03af\u03bd\u03b5\u03b9 \u03cc\u03c4\u03b9 \u03b7 \u03b5\u03bd\u03c3\u03c9\u03bc\u03ac\u03c4\u03c9\u03c3\u03b7 \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03c3\u03c7\u03b5\u03b4\u03b9\u03b1\u03c3\u03c4\u03b5\u03af \u03bc\u03b1\u03b6\u03af \u03bc\u03b5 \u03c4\u03bf framework \u03ba\u03b1\u03b9 \u03c4\u03b7 \u03c3\u03c4\u03c1\u03b1\u03c4\u03b7\u03b3\u03b9\u03ba\u03ae \u03ba\u03b1\u03c4\u03b1\u03bd\u03bf\u03bc\u03ae\u03c2 \u03bc\u03bd\u03ae\u03bc\u03b7\u03c2, \u03cc\u03c7\u03b9 \u03bd\u03b1 \u03c0\u03c1\u03bf\u03c3\u03c4\u03b5\u03b8\u03b5\u03af \u03c3\u03c4\u03bf \u03c4\u03ad\u03bb\u03bf\u03c2 \u03c9\u03c2 \u03b1\u03c0\u03bb\u03cc plugin.<\/p>\n<p>\u03a4\u03bf \u03b1\u03c0\u03bf\u03b8\u03b5\u03c4\u03ae\u03c1\u03b9\u03bf \u03b4\u03b9\u03b5\u03c5\u03ba\u03c1\u03b9\u03bd\u03af\u03b6\u03b5\u03b9 \u03cc\u03c4\u03b9 \u03c4\u03bf prefetch pool \u03b5\u03af\u03bd\u03b1\u03b9 process-global \u03ba\u03b1\u03b9 \u03bc\u03bf\u03b9\u03c1\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03bc\u03b5\u03c4\u03b1\u03be\u03cd layers. \u0386\u03c1\u03b1 \u03c4\u03bf \u03c0\u03c1\u03cc\u03c3\u03b8\u03b5\u03c4\u03bf \u03ba\u03cc\u03c3\u03c4\u03bf\u03c2 \u03b1\u03c6\u03bf\u03c1\u03ac B expert weights \u03b1\u03bd\u03ac projection \u03c3\u03c5\u03bd\u03bf\u03bb\u03b9\u03ba\u03ac \u03ba\u03b1\u03b9 \u03cc\u03c7\u03b9 \u03be\u03b5\u03c7\u03c9\u03c1\u03b9\u03c3\u03c4\u03ac \u03b3\u03b9\u03b1 \u03ba\u03ac\u03b8\u03b5 layer. \u0393\u03b9\u03b1 training \u03b1\u03c0\u03b1\u03b9\u03c4\u03b5\u03af\u03c4\u03b1\u03b9 B = E\/R. \u0393\u03b9\u03b1 inference \u03b5\u03c0\u03b9\u03c4\u03c1\u03ad\u03c0\u03b5\u03c4\u03b1\u03b9 \u03bc\u03b9\u03ba\u03c1\u03cc\u03c4\u03b5\u03c1\u03bf B, \u03bc\u03b5 \u03c0\u03c1\u03bf\u03c4\u03b5\u03b9\u03bd\u03cc\u03bc\u03b5\u03bd\u03b7 \u03c4\u03b9\u03bc\u03ae 3 \u03ad\u03c9\u03c2 4 \u03c3\u03c4\u03bf README. \u0391\u03bd \u03bf\u03b9 \u03b1\u03c0\u03bf\u03bc\u03b1\u03ba\u03c1\u03c5\u03c3\u03bc\u03ad\u03bd\u03bf\u03b9 experts \u03be\u03b5\u03c0\u03b5\u03c1\u03ac\u03c3\u03bf\u03c5\u03bd \u03c4\u03b9\u03c2 \u03b4\u03b9\u03b1\u03b8\u03ad\u03c3\u03b9\u03bc\u03b5\u03c2 \u03b8\u03ad\u03c3\u03b5\u03b9\u03c2, \u03b7 \u03b1\u03bd\u03ac\u03b3\u03bd\u03c9\u03c3\u03b7 \u03b3\u03af\u03bd\u03b5\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc \u03c4\u03bf\u03bd home rank, \u03bc\u03b5 \u03c0\u03b9\u03b8\u03b1\u03bd\u03ae \u03b5\u03c0\u03b9\u03b2\u03c1\u03ac\u03b4\u03c5\u03bd\u03c3\u03b7 \u03b1\u03bb\u03bb\u03ac \u03c7\u03c9\u03c1\u03af\u03c2 \u03b1\u03bb\u03bb\u03b1\u03b3\u03ae \u03c3\u03c4\u03b7\u03bd \u03bf\u03c1\u03b8\u03cc\u03c4\u03b7\u03c4\u03b1, \u03c3\u03cd\u03bc\u03c6\u03c9\u03bd\u03b1 \u03bc\u03b5 \u03c4\u03bf\u03c5\u03c2 \u03b4\u03b7\u03bc\u03b9\u03bf\u03c5\u03c1\u03b3\u03bf\u03cd\u03c2.<\/p>\n<h2 id=\"gradients-ekpaidefsi\">\u03a4\u03b9 \u03c3\u03c5\u03bc\u03b2\u03b1\u03af\u03bd\u03b5\u03b9 \u03c3\u03c4\u03bf\u03c5\u03c2 gradients \u03ba\u03b1\u03c4\u03ac \u03c4\u03b7\u03bd \u03b5\u03ba\u03c0\u03b1\u03af\u03b4\u03b5\u03c5\u03c3\u03b7<\/h2>\n<p>\u0397 \u03b5\u03ba\u03c0\u03b1\u03af\u03b4\u03b5\u03c5\u03c3\u03b7 \u03b1\u03c0\u03b1\u03b9\u03c4\u03b5\u03af \u03be\u03b5\u03c7\u03c9\u03c1\u03b9\u03c3\u03c4\u03ae \u03b4\u03b9\u03b1\u03c7\u03b5\u03af\u03c1\u03b9\u03c3\u03b7 \u03c4\u03c9\u03bd gradients \u03c4\u03c9\u03bd duplicated experts. \u03a4\u03bf MoonEP \u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03b5\u03af fp32 grad buffer \u03bc\u03b5 \u03b1\u03bd\u03c4\u03af\u03c3\u03c4\u03bf\u03b9\u03c7\u03b7 \u03b4\u03b9\u03ac\u03c4\u03b1\u03be\u03b7 [E+B, H, H&#8217;]. \u039f\u03b9 \u03b8\u03ad\u03c3\u03b5\u03b9\u03c2 \u03c4\u03c9\u03bd prefetched experts \u03b4\u03b5\u03bd \u03b3\u03c1\u03ac\u03c6\u03bf\u03c5\u03bd \u03b1\u03c0\u03b5\u03c5\u03b8\u03b5\u03af\u03b1\u03c2 \u03c3\u03c4\u03b1 \u03ba\u03b1\u03bd\u03bf\u03bd\u03b9\u03ba\u03ac parameter gradients. \u03a7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03bf\u03cd\u03bd \u03be\u03b5\u03c7\u03c9\u03c1\u03b9\u03c3\u03c4\u03cc reduce buffer, \u03ce\u03c3\u03c4\u03b5 \u03bf\u03b9 \u03c0\u03c1\u03bf\u03c3\u03c9\u03c1\u03b9\u03bd\u03bf\u03af gradients \u03bd\u03b1 \u03bc\u03b7\u03bd \u03b5\u03bc\u03c0\u03bb\u03b1\u03ba\u03bf\u03cd\u03bd \u03bb\u03b1\u03bd\u03b8\u03b1\u03c3\u03bc\u03ad\u03bd\u03b1 \u03c3\u03c4\u03bf gradient reduction \u03c4\u03bf\u03c5 framework.<\/p>\n<p>\u039a\u03ac\u03b8\u03b5 rank \u03c7\u03b1\u03c1\u03c4\u03bf\u03b3\u03c1\u03b1\u03c6\u03b5\u03af \u03c4\u03b1 reduce buffers \u03c4\u03c9\u03bd R ranks \u03c9\u03c2 \u03bc\u03af\u03b1 \u03ba\u03bf\u03b9\u03bd\u03ae \u03c0\u03c1\u03bf\u03b2\u03bf\u03bb\u03ae [R, B, H, H&#8217;]. \u0397 \u03bb\u03b5\u03b9\u03c4\u03bf\u03c5\u03c1\u03b3\u03af\u03b1 reduce_grad \u03b4\u03b9\u03b1\u03b2\u03ac\u03b6\u03b5\u03b9 \u03bc\u03ad\u03c3\u03c9 NVLink \u03c4\u03b9\u03c2 \u03b8\u03ad\u03c3\u03b5\u03b9\u03c2 \u03c0\u03bf\u03c5 \u03b1\u03c6\u03bf\u03c1\u03bf\u03cd\u03bd \u03c4\u03bf\u03c5\u03c2 \u03b4\u03b9\u03ba\u03bf\u03cd\u03c2 \u03c4\u03bf\u03c5 experts, \u03c0\u03c1\u03bf\u03c3\u03b8\u03ad\u03c4\u03b5\u03b9 \u03c4\u03b1 \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03ad\u03c3\u03bc\u03b1\u03c4\u03b1 \u03c3\u03c4\u03bf \u03c4\u03bf\u03c0\u03b9\u03ba\u03cc parameter gradient \u03ba\u03b1\u03b9 \u03ba\u03b1\u03b8\u03b1\u03c1\u03af\u03b6\u03b5\u03b9 \u03c4\u03b9\u03c2 \u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03b7\u03bc\u03ad\u03bd\u03b5\u03c2 \u03b8\u03ad\u03c3\u03b5\u03b9\u03c2 \u03c0\u03c1\u03b9\u03bd \u03b1\u03c0\u03cc \u03c4\u03bf \u03b5\u03c0\u03cc\u03bc\u03b5\u03bd\u03bf microbatch.<\/p>\n<p>\u0393\u03b9\u03b1 \u03ad\u03bd\u03b1\u03bd CTO \u03ae infrastructure lead, \u03b1\u03c5\u03c4\u03cc \u03c4\u03bf \u03c3\u03b7\u03bc\u03b5\u03af\u03bf \u03b5\u03af\u03bd\u03b1\u03b9 \u03ba\u03c1\u03af\u03c3\u03b9\u03bc\u03bf: \u03b7 \u03b1\u03c0\u03bf\u03b4\u03bf\u03c4\u03b9\u03ba\u03cc\u03c4\u03b7\u03c4\u03b1 \u03c4\u03b7\u03c2 \u03b5\u03c0\u03b9\u03ba\u03bf\u03b9\u03bd\u03c9\u03bd\u03af\u03b1\u03c2 \u03c3\u03c5\u03bd\u03b4\u03ad\u03b5\u03c4\u03b1\u03b9 \u03bc\u03b5 \u03b1\u03c5\u03c3\u03c4\u03b7\u03c1\u03ad\u03c2 \u03c0\u03b1\u03c1\u03b1\u03b4\u03bf\u03c7\u03ad\u03c2 \u03b3\u03b9\u03b1 \u03c4\u03bf hardware, \u03c4\u03b7 symmetric memory \u03ba\u03b1\u03b9 \u03c4\u03bf lifecycle \u03c4\u03c9\u03bd buffers. \u0397 \u03b2\u03b9\u03b2\u03bb\u03b9\u03bf\u03b8\u03ae\u03ba\u03b7 \u03b1\u03bd\u03b1\u03c6\u03ad\u03c1\u03b5\u03b9 \u03c5\u03c0\u03bf\u03c3\u03c4\u03ae\u03c1\u03b9\u03be\u03b7 NVIDIA GPU, \u03b5\u03bd\u03ce \u03b7 \u03c5\u03c0\u03bf\u03c3\u03c4\u03ae\u03c1\u03b9\u03be\u03b7 Zhenwu PPU \u03b5\u03bc\u03c6\u03b1\u03bd\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 \u03c9\u03c2 \u03c5\u03c0\u03cc \u03b5\u03be\u03ad\u03c4\u03b1\u03c3\u03b7. \u0397 \u03b4\u03bf\u03ba\u03b9\u03bc\u03ae \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03b5\u03c0\u03bf\u03bc\u03ad\u03bd\u03c9\u03c2 \u03bd\u03b1 \u03b3\u03af\u03bd\u03b5\u03b9 \u03c3\u03c4\u03bf \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc cluster \u03ba\u03b1\u03b9 \u03cc\u03c7\u03b9 \u03bc\u03cc\u03bd\u03bf \u03c3\u03b5 \u03b5\u03c0\u03af\u03c0\u03b5\u03b4\u03bf API.<\/p>\n<h2 id=\"benchmarks-oria-apodeixeon\">\u03a4\u03b9 \u03b4\u03b5\u03af\u03c7\u03bd\u03bf\u03c5\u03bd \u2014 \u03ba\u03b1\u03b9 \u03c4\u03b9 \u03b4\u03b5\u03bd \u03b1\u03c0\u03bf\u03b4\u03b5\u03b9\u03ba\u03bd\u03cd\u03bf\u03c5\u03bd \u2014 \u03c4\u03b1 benchmarks<\/h2>\n<p>\u03a4\u03bf \u03b4\u03b7\u03bc\u03cc\u03c3\u03b9\u03bf benchmark \u03c3\u03c5\u03b3\u03ba\u03c1\u03af\u03bd\u03b5\u03b9 \u03c4\u03bf MoonEP \u03bc\u03b5 \u03c4\u03bf DeepEP v2 \u03c3\u03b5 NVIDIA H20 \u03bc\u03b5 expert parallelism 8. \u03a4\u03bf script \u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03b5\u03af \u03c0\u03c1\u03bf\u03b5\u03c0\u03b9\u03bb\u03bf\u03b3\u03ad\u03c2 S=8192, E=384, H=7168, K=8, H&#8217;=2048 \u03ba\u03b1\u03b9 32 SMs, \u03b5\u03bd\u03ce \u03b5\u03be\u03b5\u03c4\u03ac\u03b6\u03b5\u03b9 \u03c4\u03b9\u03c2 \u03c0\u03c1\u03bf\u03b5\u03c0\u03b9\u03bb\u03b5\u03b3\u03bc\u03ad\u03bd\u03b5\u03c2 \u03c4\u03b9\u03bc\u03ad\u03c2 maxvio 0,2, 1, 10 \u03ba\u03b1\u03b9 20. \u039f\u03b9 \u03b4\u03cd\u03bf \u03b2\u03b9\u03b2\u03bb\u03b9\u03bf\u03b8\u03ae\u03ba\u03b5\u03c2 \u03bb\u03b1\u03bc\u03b2\u03ac\u03bd\u03bf\u03c5\u03bd \u03c4\u03b7\u03bd \u03af\u03b4\u03b9\u03b1 routing matrix \u03b1\u03c0\u03cc \u03ba\u03bf\u03b9\u03bd\u03cc seed.<\/p>\n<div class=\"td-chart td-chart--metrics\">\n<p class=\"td-chart-title\">\u0397 \u03c0\u03c1\u03bf\u03b5\u03c0\u03b9\u03bb\u03b5\u03b3\u03bc\u03ad\u03bd\u03b7 \u03c1\u03cd\u03b8\u03bc\u03b9\u03c3\u03b7 \u03c4\u03bf\u03c5 \u03b4\u03b7\u03bc\u03cc\u03c3\u03b9\u03bf\u03c5 benchmark<\/p>\n<p class=\"td-chart-intro\">\u039f\u03b9 \u03c4\u03b9\u03bc\u03ad\u03c2 \u03c0\u03c1\u03bf\u03ad\u03c1\u03c7\u03bf\u03bd\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc \u03c4\u03bf script bench_vs_deepep.py \u03ba\u03b1\u03b9 \u03c0\u03b5\u03c1\u03b9\u03b3\u03c1\u03ac\u03c6\u03bf\u03c5\u03bd \u03c4\u03bf \u03c3\u03c5\u03b3\u03ba\u03b5\u03ba\u03c1\u03b9\u03bc\u03ad\u03bd\u03bf test \u03c3\u03b5 H20. \u0394\u03b5\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03c0\u03c1\u03bf\u03c4\u03b5\u03b9\u03bd\u03cc\u03bc\u03b5\u03bd\u03b7 \u03b4\u03b9\u03b1\u03bc\u03cc\u03c1\u03c6\u03c9\u03c3\u03b7 \u03b3\u03b9\u03b1 \u03ba\u03ac\u03b8\u03b5 cluster \u03bf\u03cd\u03c4\u03b5 \u03b1\u03bd\u03b5\u03be\u03ac\u03c1\u03c4\u03b7\u03c4\u03b7 \u03bc\u03ad\u03c4\u03c1\u03b7\u03c3\u03b7 \u03b1\u03c0\u03cc \u03c4\u03b7\u03bd TWO DOTS.<\/p>\n<div class=\"td-metric-grid\">\n<div class=\"td-metric-card\"><span class=\"td-metric-value\">8<\/span><strong>EP ranks<\/strong><\/p>\n<p>\u03a4\u03bf README \u03b4\u03b7\u03bb\u03ce\u03bd\u03b5\u03b9 expert parallelism \u03bf\u03ba\u03c4\u03ce ranks \u03b3\u03b9\u03b1 \u03c4\u03b1 \u03b4\u03b7\u03bc\u03bf\u03c3\u03b9\u03b5\u03c5\u03bc\u03ad\u03bd\u03b1 benchmarks.<\/p>\n<\/div>\n<div class=\"td-metric-card\"><span class=\"td-metric-value\">8192<\/span><strong>tokens \u03b1\u03bd\u03ac rank<\/strong><\/p>\n<p>\u0397 \u03c0\u03c1\u03bf\u03b5\u03c0\u03b9\u03bb\u03b5\u03b3\u03bc\u03ad\u03bd\u03b7 \u03c4\u03b9\u03bc\u03ae S \u03c4\u03bf\u03c5 script \u03b3\u03b9\u03b1 \u03ba\u03ac\u03b8\u03b5 rank \u03c0\u03c1\u03b9\u03bd \u03b1\u03c0\u03cc \u03c4\u03bf\u03bd \u03c0\u03bf\u03bb\u03bb\u03b1\u03c0\u03bb\u03b1\u03c3\u03b9\u03b1\u03c3\u03bc\u03cc \u03bc\u03b5 top-k.<\/p>\n<\/div>\n<div class=\"td-metric-card\"><span class=\"td-metric-value\">384<\/span><strong>routed experts<\/strong><\/p>\n<p>\u0397 \u03c0\u03c1\u03bf\u03b5\u03c0\u03b9\u03bb\u03b5\u03b3\u03bc\u03ad\u03bd\u03b7 \u03c4\u03b9\u03bc\u03ae E \u03c3\u03c4\u03b7 \u03c3\u03cd\u03b3\u03ba\u03c1\u03b9\u03c3\u03b7 MoonEP \u03ba\u03b1\u03b9 DeepEP v2.<\/p>\n<\/div>\n<div class=\"td-metric-card\"><span class=\"td-metric-value\">8<\/span><strong>top-k experts<\/strong><\/p>\n<p>\u0397 \u03c0\u03c1\u03bf\u03b5\u03c0\u03b9\u03bb\u03b5\u03b3\u03bc\u03ad\u03bd\u03b7 \u03c4\u03b9\u03bc\u03ae K, \u03bc\u03b5 \u03b4\u03bf\u03ba\u03b9\u03bc\u03ad\u03c2 maxvio 0,2, 1, 10 \u03ba\u03b1\u03b9 20.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<p>\u039f\u03b9 \u03b4\u03b7\u03bc\u03b9\u03bf\u03c5\u03c1\u03b3\u03bf\u03af \u03b1\u03bd\u03b1\u03c6\u03ad\u03c1\u03bf\u03c5\u03bd \u03cc\u03c4\u03b9 \u03c4\u03bf MoonEP \u03b4\u03b9\u03b1\u03c4\u03b7\u03c1\u03b5\u03af \u03c3\u03c7\u03b5\u03b4\u03cc\u03bd \u03c3\u03c4\u03b1\u03b8\u03b5\u03c1\u03cc communication time \u03ba\u03b1\u03b8\u03ce\u03c2 \u03b1\u03c5\u03be\u03ac\u03bd\u03b5\u03c4\u03b1\u03b9 \u03b7 \u03b1\u03bd\u03b9\u03c3\u03bf\u03c1\u03c1\u03bf\u03c0\u03af\u03b1, \u03b5\u03bd\u03ce \u03c4\u03bf DeepEP v2 \u03b5\u03c0\u03b9\u03b2\u03c1\u03b1\u03b4\u03cd\u03bd\u03b5\u03c4\u03b1\u03b9. \u03a5\u03c0\u03bf\u03c3\u03c4\u03b7\u03c1\u03af\u03b6\u03bf\u03c5\u03bd \u03b5\u03c0\u03af\u03c3\u03b7\u03c2 \u03cc\u03c4\u03b9 \u03c4\u03bf zero copy \u03bc\u03b5\u03b9\u03ce\u03bd\u03b5\u03b9 \u03c4\u03b7\u03bd \u03b5\u03c0\u03b9\u03ba\u03bf\u03b9\u03bd\u03c9\u03bd\u03af\u03b1 \u03ba\u03b1\u03b9 \u03cc\u03c4\u03b9 \u03c4\u03b1 \u03c3\u03c4\u03b1\u03c4\u03b9\u03ba\u03ac shapes \u03b1\u03c0\u03bf\u03c6\u03b5\u03cd\u03b3\u03bf\u03c5\u03bd fragmentation \u03ba\u03b1\u03b9 out-of-memory \u03c3\u03b5 \u03c5\u03c8\u03b7\u03bb\u03ae \u03b1\u03bd\u03b9\u03c3\u03bf\u03c1\u03c1\u03bf\u03c0\u03af\u03b1. \u0391\u03c5\u03c4\u03ac \u03b5\u03af\u03bd\u03b1\u03b9 \u03b4\u03b7\u03bc\u03bf\u03c3\u03b9\u03b5\u03c5\u03bc\u03ad\u03bd\u03b1 \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03ad\u03c3\u03bc\u03b1\u03c4\u03b1 \u03c4\u03bf\u03c5 \u03b1\u03c0\u03bf\u03b8\u03b5\u03c4\u03b7\u03c1\u03af\u03bf\u03c5, \u03cc\u03c7\u03b9 \u03b1\u03bd\u03b5\u03be\u03ac\u03c1\u03c4\u03b7\u03c4\u03b7 \u03b1\u03bd\u03b1\u03c0\u03b1\u03c1\u03b1\u03b3\u03c9\u03b3\u03ae \u03b1\u03c0\u03cc \u03c4\u03b7\u03bd TWO DOTS. \u0397 \u03b4\u03b9\u03ac\u03ba\u03c1\u03b9\u03c3\u03b7 \u03b1\u03bd\u03ac\u03bc\u03b5\u03c3\u03b1 \u03c3\u03b5 \u03bc\u03b5\u03c4\u03c1\u03ae\u03c3\u03b5\u03b9\u03c2 \u03c4\u03c9\u03bd \u03b4\u03b7\u03bc\u03b9\u03bf\u03c5\u03c1\u03b3\u03ce\u03bd \u03ba\u03b1\u03b9 \u03c3\u03b5 \u03b5\u03be\u03c9\u03c4\u03b5\u03c1\u03b9\u03ba\u03ae \u03b5\u03c0\u03b9\u03b2\u03b5\u03b2\u03b1\u03af\u03c9\u03c3\u03b7 \u03b9\u03c3\u03c7\u03cd\u03b5\u03b9 \u03c3\u03b5 \u03ba\u03ac\u03b8\u03b5 <a href=\"https:\/\/twodots.gr\/rl-sft-reasoning-models-esoterika-representations\/\">benchmark \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03c9\u03bd \u03c3\u03c5\u03bb\u03bb\u03bf\u03b3\u03b9\u03c3\u03bc\u03bf\u03cd<\/a>.<\/p>\n<p>\u0394\u03b5\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03c3\u03c9\u03c3\u03c4\u03cc \u03bd\u03b1 \u03bc\u03b5\u03c4\u03b1\u03c4\u03c1\u03b1\u03c0\u03bf\u03cd\u03bd \u03b1\u03c5\u03c4\u03cc\u03bc\u03b1\u03c4\u03b1 \u03bf\u03b9 \u03c3\u03c5\u03b3\u03ba\u03b5\u03ba\u03c1\u03b9\u03bc\u03ad\u03bd\u03b5\u03c2 \u03bc\u03b5\u03c4\u03c1\u03ae\u03c3\u03b5\u03b9\u03c2 \u03c3\u03b5 \u03c0\u03c1\u03cc\u03b2\u03bb\u03b5\u03c8\u03b7 \u03ba\u03cc\u03c3\u03c4\u03bf\u03c5\u03c2 \u03b3\u03b9\u03b1 \u03ba\u03ac\u03b8\u03b5 cluster. \u0397 \u03b1\u03c0\u03cc\u03b4\u03bf\u03c3\u03b7 \u03b5\u03be\u03b1\u03c1\u03c4\u03ac\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc GPU, NVLink topology, model architecture, routing distribution, batch profile, framework integration \u03ba\u03b1\u03b9 training configuration. \u03a4\u03bf benchmark \u03b5\u03af\u03bd\u03b1\u03b9 \u03b9\u03c3\u03c7\u03c5\u03c1\u03ae \u03ad\u03bd\u03b4\u03b5\u03b9\u03be\u03b7 \u03b3\u03b9\u03b1 \u03c0\u03b5\u03c1\u03b1\u03b9\u03c4\u03ad\u03c1\u03c9 \u03b4\u03bf\u03ba\u03b9\u03bc\u03ae, \u03cc\u03c7\u03b9 \u03b5\u03b3\u03b3\u03cd\u03b7\u03c3\u03b7 \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03b7\u03bc\u03b1\u03c4\u03b9\u03ba\u03bf\u03cd ROI.<\/p>\n<h2 id=\"epicheiriseis-ai-ypodomi\">\u0393\u03b9\u03b1\u03c4\u03af \u03b5\u03bd\u03b4\u03b9\u03b1\u03c6\u03ad\u03c1\u03b5\u03b9 \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03ae\u03c3\u03b5\u03b9\u03c2 \u03c0\u03bf\u03c5 \u03b1\u03b3\u03bf\u03c1\u03ac\u03b6\u03bf\u03c5\u03bd \u03ae \u03c7\u03c4\u03af\u03b6\u03bf\u03c5\u03bd AI<\/h2>\n<p>\u0393\u03b9\u03b1 \u03c4\u03b9\u03c2 \u03c0\u03b5\u03c1\u03b9\u03c3\u03c3\u03cc\u03c4\u03b5\u03c1\u03b5\u03c2 \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03ae\u03c3\u03b5\u03b9\u03c2, \u03c4\u03bf MoonEP \u03b4\u03b5\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03b5\u03c1\u03b3\u03b1\u03bb\u03b5\u03af\u03bf marketing \u03ae e-commerce \u03bc\u03b5 \u03ac\u03bc\u03b5\u03c3\u03b7 \u03c7\u03c1\u03ae\u03c3\u03b7. \u0395\u03af\u03bd\u03b1\u03b9 \u03cc\u03bc\u03c9\u03c2 \u03c0\u03b1\u03c1\u03ac\u03b4\u03b5\u03b9\u03b3\u03bc\u03b1 \u03c4\u03bf\u03c5 \u03c0\u03ce\u03c2 \u03b7 \u03c5\u03c0\u03bf\u03b4\u03bf\u03bc\u03ae \u03b5\u03c0\u03b7\u03c1\u03b5\u03ac\u03b6\u03b5\u03b9 \u03c4\u03bf \u03ba\u03cc\u03c3\u03c4\u03bf\u03c2, \u03c4\u03b7 \u03c3\u03c4\u03b1\u03b8\u03b5\u03c1\u03cc\u03c4\u03b7\u03c4\u03b1 \u03ba\u03b1\u03b9 \u03c4\u03b7\u03bd \u03c4\u03b1\u03c7\u03cd\u03c4\u03b7\u03c4\u03b1 \u03b5\u03bd\u03cc\u03c2 AI \u03c0\u03c1\u03bf\u03ca\u03cc\u03bd\u03c4\u03bf\u03c2. \u038c\u03c4\u03b1\u03bd \u03ad\u03bd\u03b1\u03c2 \u03c0\u03ac\u03c1\u03bf\u03c7\u03bf\u03c2 \u03b9\u03c3\u03c7\u03c5\u03c1\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 \u03cc\u03c4\u03b9 \u03ad\u03bd\u03b1 \u03bc\u03b5\u03b3\u03ac\u03bb\u03bf MoE \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03bf \u03b5\u03af\u03bd\u03b1\u03b9 \u03bf\u03b9\u03ba\u03bf\u03bd\u03bf\u03bc\u03b9\u03ba\u03cc\u03c4\u03b5\u03c1\u03bf, \u03b7 \u03c3\u03c9\u03c3\u03c4\u03ae \u03b5\u03c1\u03ce\u03c4\u03b7\u03c3\u03b7 \u03b4\u03b5\u03bd \u03b1\u03c6\u03bf\u03c1\u03ac \u03bc\u03cc\u03bd\u03bf \u03c0\u03cc\u03c3\u03b5\u03c2 \u03c0\u03b1\u03c1\u03b1\u03bc\u03ad\u03c4\u03c1\u03bf\u03c5\u03c2 \u03ad\u03c7\u03b5\u03b9. \u0391\u03c6\u03bf\u03c1\u03ac \u03ba\u03b1\u03b9 \u03c4\u03bf \u03c0\u03ce\u03c2 \u03b4\u03c1\u03bf\u03bc\u03bf\u03bb\u03bf\u03b3\u03bf\u03cd\u03bd\u03c4\u03b1\u03b9 \u03c4\u03b1 tokens, \u03c0\u03cc\u03c3\u03bf \u03c3\u03c5\u03c7\u03bd\u03ac \u03c0\u03b5\u03c1\u03b9\u03bc\u03ad\u03bd\u03bf\u03c5\u03bd \u03bf\u03b9 accelerators \u03ba\u03b1\u03b9 \u03b1\u03bd \u03b7 \u03bc\u03bd\u03ae\u03bc\u03b7 \u03c0\u03b1\u03c1\u03b1\u03bc\u03ad\u03bd\u03b5\u03b9 \u03c0\u03c1\u03bf\u03b2\u03bb\u03ad\u03c8\u03b9\u03bc\u03b7.<\/p>\n<p>\u039f\u03bc\u03ac\u03b4\u03b5\u03c2 \u03c0\u03bf\u03c5 \u03b1\u03bd\u03b1\u03c0\u03c4\u03cd\u03c3\u03c3\u03bf\u03c5\u03bd \u03b9\u03b4\u03b9\u03c9\u03c4\u03b9\u03ba\u03ac \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03b1 \u03ae \u03b5\u03be\u03b5\u03b9\u03b4\u03b9\u03ba\u03b5\u03c5\u03bc\u03ad\u03bd\u03b1 inference endpoints \u03bc\u03c0\u03bf\u03c1\u03bf\u03cd\u03bd \u03bd\u03b1 \u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03ae\u03c3\u03bf\u03c5\u03bd \u03c4\u03b9\u03c2 \u03b1\u03c1\u03c7\u03ad\u03c2 \u03c4\u03bf\u03c5 MoonEP \u03c9\u03c2 checklist \u03b1\u03be\u03b9\u03bf\u03bb\u03cc\u03b3\u03b7\u03c3\u03b7\u03c2. \u03a5\u03c0\u03ac\u03c1\u03c7\u03b5\u03b9 \u03c3\u03c4\u03b1\u03b8\u03b5\u03c1\u03cc \u03c6\u03bf\u03c1\u03c4\u03af\u03bf \u03b1\u03bd\u03ac rank; \u03a0\u03bf\u03b9\u03bf \u03b5\u03af\u03bd\u03b1\u03b9 \u03c4\u03bf \u03ba\u03cc\u03c3\u03c4\u03bf\u03c2 prefetch; \u03a0\u03ce\u03c2 \u03b5\u03c0\u03b7\u03c1\u03b5\u03ac\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 \u03c4\u03b1 gradients; \u03a5\u03c0\u03ac\u03c1\u03c7\u03bf\u03c5\u03bd \u03bc\u03b5\u03c4\u03c1\u03ae\u03c3\u03b5\u03b9\u03c2 \u03bc\u03b5 \u03c4\u03b7 \u03b4\u03b9\u03ba\u03ae \u03bc\u03b1\u03c2 \u03ba\u03b1\u03c4\u03b1\u03bd\u03bf\u03bc\u03ae requests; \u03a4\u03b9 \u03c3\u03c5\u03bc\u03b2\u03b1\u03af\u03bd\u03b5\u03b9 \u03cc\u03c4\u03b1\u03bd \u03c4\u03bf routing \u03b3\u03af\u03bd\u03b5\u03b9 \u03ad\u03bd\u03c4\u03bf\u03bd\u03b1 \u03ac\u03bd\u03b9\u03c3\u03bf;<\/p>\n<p>\u0393\u03b9\u03b1 marketers \u03ba\u03b1\u03b9 e-commerce owners, \u03b7 \u03c0\u03c1\u03b1\u03ba\u03c4\u03b9\u03ba\u03ae \u03c3\u03c5\u03bd\u03ad\u03c0\u03b5\u03b9\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 \u03ad\u03bc\u03bc\u03b5\u03c3\u03b7 \u03b1\u03bb\u03bb\u03ac \u03bf\u03c5\u03c3\u03b9\u03b1\u03c3\u03c4\u03b9\u03ba\u03ae. \u0397 \u03c0\u03bf\u03b9\u03cc\u03c4\u03b7\u03c4\u03b1 \u03b5\u03bd\u03cc\u03c2 <a href=\"https:\/\/twodots.gr\/pos-na-aytomatopoiisete-ergasies-me-techniti-noimosyni\/\">AI workflow \u03c0\u03bf\u03c5 \u03b1\u03c5\u03c4\u03bf\u03bc\u03b1\u03c4\u03bf\u03c0\u03bf\u03b9\u03b5\u03af \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b5\u03c2<\/a> \u03b4\u03b5\u03bd \u03ba\u03b1\u03b8\u03bf\u03c1\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 \u03bc\u03cc\u03bd\u03bf \u03b1\u03c0\u03cc \u03c4\u03bf prompt. \u0397 \u03b1\u03c0\u03cc\u03ba\u03c1\u03b9\u03c3\u03b7, \u03c4\u03bf capacity \u03ba\u03b1\u03b9 \u03c4\u03bf \u03ba\u03cc\u03c3\u03c4\u03bf\u03c2 \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b5\u03be\u03b1\u03c1\u03c4\u03ce\u03bd\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc \u03c7\u03b1\u03bc\u03b7\u03bb\u03bf\u03cd \u03b5\u03c0\u03b9\u03c0\u03ad\u03b4\u03bf\u03c5 \u03b1\u03c0\u03bf\u03c6\u03ac\u03c3\u03b5\u03b9\u03c2 \u03c0\u03bf\u03c5 \u03bf \u03c4\u03b5\u03bb\u03b9\u03ba\u03cc\u03c2 \u03c7\u03c1\u03ae\u03c3\u03c4\u03b7\u03c2 \u03b4\u03b5\u03bd \u03b2\u03bb\u03ad\u03c0\u03b5\u03b9. \u0393\u03b9\u2019 \u03b1\u03c5\u03c4\u03cc \u03bf\u03b9 \u03c0\u03c1\u03bf\u03bc\u03ae\u03b8\u03b5\u03b9\u03b5\u03c2 AI \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 \u03c4\u03b5\u03c7\u03bd\u03b9\u03ba\u03ac SLAs \u03ba\u03b1\u03b9 \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03ad\u03c2 \u03b4\u03bf\u03ba\u03b9\u03bc\u03ad\u03c2 \u03c6\u03bf\u03c1\u03c4\u03af\u03bf\u03c5, \u03cc\u03c7\u03b9 \u03bc\u03cc\u03bd\u03bf demo \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03ad\u03c3\u03bc\u03b1\u03c4\u03b1.<\/p>\n<div class=\"td-decision-band\">\n<p class=\"td-decision-label\">\u03a4\u03bf \u03ba\u03c1\u03b9\u03c4\u03ae\u03c1\u03b9\u03bf \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03b7\u03bc\u03b1\u03c4\u03b9\u03ba\u03ae\u03c2 \u03b1\u03c0\u03cc\u03c6\u03b1\u03c3\u03b7\u03c2<\/p>\n<p><strong>\u03a4\u03bf \u03c7\u03b1\u03bc\u03b7\u03bb\u03cc\u03c4\u03b5\u03c1\u03bf communication time \u03ad\u03c7\u03b5\u03b9 \u03b1\u03be\u03af\u03b1 \u03bc\u03cc\u03bd\u03bf \u03b1\u03bd \u03b5\u03c0\u03b9\u03b2\u03b5\u03b2\u03b1\u03b9\u03ce\u03bd\u03b5\u03c4\u03b1\u03b9 \u03c3\u03c4\u03bf \u03b4\u03b9\u03ba\u03cc \u03c3\u03b1\u03c2 AI workload.<\/strong><\/p>\n<p>\u0396\u03b7\u03c4\u03ae\u03c3\u03c4\u03b5 \u03b1\u03c0\u03cc \u03c4\u03bf\u03bd \u03c0\u03ac\u03c1\u03bf\u03c7\u03bf \u03bd\u03b1 \u03b4\u03b5\u03af\u03be\u03b5\u03b9 p95 \u03c7\u03c1\u03cc\u03bd\u03bf iteration \u03ae \u03b1\u03c0\u03cc\u03ba\u03c1\u03b9\u03c3\u03b7\u03c2, peak \u03bc\u03bd\u03ae\u03bc\u03b7, \u03c3\u03c5\u03bc\u03c0\u03b5\u03c1\u03b9\u03c6\u03bf\u03c1\u03ac \u03c3\u03b5 routing skew, \u03ba\u03cc\u03c3\u03c4\u03bf\u03c2 prefetch, \u03b1\u03c0\u03bf\u03c4\u03c5\u03c7\u03af\u03b5\u03c2 OOM \u03ba\u03b1\u03b9 \u03b4\u03b9\u03b1\u03b4\u03b9\u03ba\u03b1\u03c3\u03af\u03b1 fallback. \u03a7\u03c9\u03c1\u03af\u03c2 \u03b1\u03c5\u03c4\u03ac, \u03c4\u03bf benchmark \u03c0\u03b1\u03c1\u03b1\u03bc\u03ad\u03bd\u03b5\u03b9 \u03c4\u03b5\u03c7\u03bd\u03b9\u03ba\u03ae \u03ad\u03bd\u03b4\u03b5\u03b9\u03be\u03b7 \u03ba\u03b1\u03b9 \u03cc\u03c7\u03b9 \u03c4\u03b5\u03ba\u03bc\u03b7\u03c1\u03b9\u03c9\u03bc\u03ad\u03bd\u03bf SLA.<\/p>\n<\/div>\n<h2 id=\"plaisio-axiologisis-yiothetisis\">\u0388\u03bd\u03b1 \u03c1\u03b5\u03b1\u03bb\u03b9\u03c3\u03c4\u03b9\u03ba\u03cc \u03c0\u03bb\u03b1\u03af\u03c3\u03b9\u03bf \u03b1\u03be\u03b9\u03bf\u03bb\u03cc\u03b3\u03b7\u03c3\u03b7\u03c2 \u03c0\u03c1\u03b9\u03bd \u03b1\u03c0\u03cc \u03c4\u03b7\u03bd \u03c5\u03b9\u03bf\u03b8\u03ad\u03c4\u03b7\u03c3\u03b7<\/h2>\n<p>\u0397 \u03c0\u03c1\u03ce\u03c4\u03b7 \u03b4\u03bf\u03ba\u03b9\u03bc\u03ae \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03b1\u03bd\u03b1\u03c0\u03b1\u03c1\u03ac\u03b3\u03b5\u03b9 \u03c4\u03bf \u03b4\u03b9\u03ba\u03cc \u03c3\u03b1\u03c2 routing profile \u03ba\u03b1\u03b9 \u03bd\u03b1 \u03c3\u03c5\u03b3\u03ba\u03c1\u03af\u03bd\u03b5\u03b9 \u03c7\u03c1\u03cc\u03bd\u03bf iteration, \u03c7\u03c1\u03ae\u03c3\u03b7 \u03bc\u03bd\u03ae\u03bc\u03b7\u03c2 \u03ba\u03b1\u03b9 \u03c3\u03c4\u03b1\u03b8\u03b5\u03c1\u03cc\u03c4\u03b7\u03c4\u03b1 \u03bc\u03b5 \u03c4\u03b7\u03bd \u03c5\u03c0\u03ac\u03c1\u03c7\u03bf\u03c5\u03c3\u03b1 \u03c3\u03c4\u03bf\u03af\u03b2\u03b1. \u0397 \u03b4\u03b5\u03cd\u03c4\u03b5\u03c1\u03b7 \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03b5\u03bb\u03ad\u03b3\u03be\u03b5\u03b9 \u03c4\u03b7\u03bd \u03b5\u03bd\u03c3\u03c9\u03bc\u03ac\u03c4\u03c9\u03c3\u03b7: contiguous VMM ranges, \u03b4\u03b9\u03b1\u03c7\u03b5\u03af\u03c1\u03b9\u03c3\u03b7 prefetch pools, gradient buffers, zero-copy aliases \u03ba\u03b1\u03b9 \u03b1\u03c3\u03c6\u03b1\u03bb\u03ad\u03c2 lifecycle \u03c4\u03c9\u03bd views.<\/p>\n<p>\u0397 \u03c4\u03b5\u03ba\u03bc\u03b7\u03c1\u03af\u03c9\u03c3\u03b7 \u03c0\u03c1\u03bf\u03b5\u03b9\u03b4\u03bf\u03c0\u03bf\u03b9\u03b5\u03af \u03cc\u03c4\u03b9 \u03c4\u03b1 zero-copy views \u03b1\u03bd\u03c4\u03b9\u03ba\u03b1\u03b8\u03af\u03c3\u03c4\u03b1\u03bd\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc \u03c4\u03b7\u03bd \u03b5\u03c0\u03cc\u03bc\u03b5\u03bd\u03b7 \u03bb\u03b5\u03b9\u03c4\u03bf\u03c5\u03c1\u03b3\u03af\u03b1 dispatch \u03ae combine. \u0394\u03b5\u03bd \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03b4\u03b9\u03b1\u03c4\u03b7\u03c1\u03bf\u03cd\u03bd\u03c4\u03b1\u03b9 \u03c0\u03ad\u03c1\u03b1 \u03b1\u03c0\u03cc \u03b1\u03c5\u03c4\u03ad\u03c2 \u03c4\u03b9\u03c2 \u03ba\u03bb\u03ae\u03c3\u03b5\u03b9\u03c2 \u03ba\u03b1\u03b9 \u03c4\u03bf autograd \u03b4\u03b5\u03bd \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03c4\u03b1 \u03b1\u03c0\u03bf\u03b8\u03b7\u03ba\u03b5\u03cd\u03b5\u03b9 \u03b3\u03b9\u03b1 backward. \u03a3\u03b5 \u03c4\u03ad\u03c4\u03bf\u03b9\u03b1 \u03c0\u03b5\u03c1\u03af\u03c0\u03c4\u03c9\u03c3\u03b7 \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 zero_copy=false. \u0391\u03c5\u03c4\u03cc \u03b5\u03af\u03bd\u03b1\u03b9 \u03c7\u03b1\u03c1\u03b1\u03ba\u03c4\u03b7\u03c1\u03b9\u03c3\u03c4\u03b9\u03ba\u03cc \u03c0\u03b1\u03c1\u03ac\u03b4\u03b5\u03b9\u03b3\u03bc\u03b1 \u03cc\u03c0\u03bf\u03c5 \u03bc\u03b9\u03b1 \u03b2\u03b5\u03bb\u03c4\u03b9\u03c3\u03c4\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b7 \u03b1\u03c0\u03cc\u03b4\u03bf\u03c3\u03b7\u03c2 \u03b4\u03b7\u03bc\u03b9\u03bf\u03c5\u03c1\u03b3\u03b5\u03af \u03b1\u03c5\u03c3\u03c4\u03b7\u03c1\u03cc\u03c4\u03b5\u03c1\u03b7 \u03c5\u03c0\u03bf\u03c7\u03c1\u03ad\u03c9\u03c3\u03b7 \u03bf\u03c1\u03b8\u03cc\u03c4\u03b7\u03c4\u03b1\u03c2.<\/p>\n<p>\u03a4\u03ad\u03bb\u03bf\u03c2, \u03b7 \u03ac\u03b4\u03b5\u03b9\u03b1 MIT \u03b4\u03b9\u03b5\u03c5\u03ba\u03bf\u03bb\u03cd\u03bd\u03b5\u03b9 \u03c4\u03b7\u03bd \u03b5\u03bc\u03c0\u03bf\u03c1\u03b9\u03ba\u03ae \u03b1\u03be\u03b9\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b7, \u03b1\u03bb\u03bb\u03ac \u03b4\u03b5\u03bd \u03b1\u03bd\u03c4\u03b9\u03ba\u03b1\u03b8\u03b9\u03c3\u03c4\u03ac \u03c4\u03bf\u03bd \u03ad\u03bb\u03b5\u03b3\u03c7\u03bf \u03c3\u03c5\u03bd\u03c4\u03ae\u03c1\u03b7\u03c3\u03b7\u03c2, \u03c3\u03c5\u03bc\u03b2\u03b1\u03c4\u03cc\u03c4\u03b7\u03c4\u03b1\u03c2 \u03ba\u03b1\u03b9 \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03b7\u03c3\u03b9\u03b1\u03ba\u03bf\u03cd \u03c1\u03af\u03c3\u03ba\u03bf\u03c5. \u03a4\u03bf \u03b1\u03c0\u03bf\u03b8\u03b5\u03c4\u03ae\u03c1\u03b9\u03bf \u03ae\u03c4\u03b1\u03bd \u03bd\u03ad\u03bf \u03ba\u03b1\u03c4\u03ac \u03c4\u03b7 \u03b4\u03b7\u03bc\u03bf\u03c3\u03af\u03b5\u03c5\u03c3\u03b7 \u03c4\u03b7\u03c2 \u03c0\u03b7\u03b3\u03ae\u03c2. \u039c\u03b9\u03b1 \u03bf\u03bc\u03ac\u03b4\u03b1 \u03b8\u03b1 \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03b5\u03c0\u03b9\u03b2\u03b5\u03b2\u03b1\u03b9\u03ce\u03c3\u03b5\u03b9 \u03c4\u03b7\u03bd \u03b5\u03be\u03ad\u03bb\u03b9\u03be\u03b7 \u03c4\u03bf\u03c5 \u03ba\u03ce\u03b4\u03b9\u03ba\u03b1, \u03bd\u03b1 \u03c4\u03c1\u03ad\u03be\u03b5\u03b9 \u03c4\u03b1 \u03c0\u03b1\u03c1\u03b5\u03c7\u03cc\u03bc\u03b5\u03bd\u03b1 multi-GPU tests \u03ba\u03b1\u03b9 \u03bd\u03b1 \u03ba\u03b1\u03c4\u03b1\u03b3\u03c1\u03ac\u03c8\u03b5\u03b9 \u03c3\u03b1\u03c6\u03ae fallback \u03b4\u03b9\u03b1\u03b4\u03c1\u03bf\u03bc\u03ae \u03c0\u03c1\u03b9\u03bd \u03b5\u03be\u03b1\u03c1\u03c4\u03ae\u03c3\u03b5\u03b9 \u03c0\u03b1\u03c1\u03b1\u03b3\u03c9\u03b3\u03b9\u03ba\u03cc \u03c3\u03cd\u03c3\u03c4\u03b7\u03bc\u03b1 \u03b1\u03c0\u03cc \u03c4\u03b7 \u03b2\u03b9\u03b2\u03bb\u03b9\u03bf\u03b8\u03ae\u03ba\u03b7. \u0397 \u03c0\u03b1\u03c1\u03b1\u03ba\u03bf\u03bb\u03bf\u03cd\u03b8\u03b7\u03c3\u03b7 \u03ba\u03b1\u03b9 \u03b7 \u03c3\u03b1\u03c6\u03ae\u03c2 \u03b1\u03c0\u03cc\u03b4\u03bf\u03c3\u03b7 \u03b1\u03b9\u03c4\u03af\u03b1\u03c2 \u03b5\u03af\u03bd\u03b1\u03b9 \u03b1\u03c0\u03b1\u03c1\u03b1\u03af\u03c4\u03b7\u03c4\u03b5\u03c2, \u03cc\u03c0\u03c9\u03c2 \u03ba\u03b1\u03b9 \u03c3\u03b5 \u03ba\u03ac\u03b8\u03b5 <a href=\"https:\/\/twodots.gr\/ai-agents-poios-ftaiei-otan-aftomatopoiisi-apotygchanei\/\">\u03b1\u03c0\u03bf\u03c4\u03c5\u03c7\u03af\u03b1 \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03b7\u03c3\u03b9\u03b1\u03ba\u03bf\u03cd AI \u03b1\u03c5\u03c4\u03bf\u03bc\u03b1\u03c4\u03b9\u03c3\u03bc\u03bf\u03cd<\/a>.<\/p>\n<div class=\"td-step-list\">\n<p class=\"td-step-list-title\">\u0388\u03be\u03b9 \u03ad\u03bb\u03b5\u03b3\u03c7\u03bf\u03b9 \u03c0\u03c1\u03b9\u03bd \u03b1\u03c0\u03cc \u03b4\u03bf\u03ba\u03b9\u03bc\u03ae MoonEP<\/p>\n<ol>\n<li><span class=\"td-step-kicker\">\u0392\u03ae\u03bc\u03b1 1<\/span><strong>\u0391\u03c0\u03bf\u03c4\u03c5\u03c0\u03ce\u03c3\u03c4\u03b5 \u03c4\u03bf \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc routing profile<\/strong>\n<p>\u039c\u03b5\u03c4\u03c1\u03ae\u03c3\u03c4\u03b5 tokens \u03b1\u03bd\u03ac expert, maxvio \u03ba\u03b1\u03b9 \u03c4\u03b7 \u03c3\u03c5\u03c7\u03bd\u03cc\u03c4\u03b7\u03c4\u03b1 \u03c4\u03c9\u03bd hot experts \u03c3\u03c4\u03bf \u03b4\u03b9\u03ba\u03cc \u03c3\u03b1\u03c2 \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03bf \u03ba\u03b1\u03b9 \u03c3\u03c4\u03b1 \u03b4\u03b9\u03ba\u03ac \u03c3\u03b1\u03c2 batches.<\/p>\n<\/li>\n<li><span class=\"td-step-kicker\">\u0392\u03ae\u03bc\u03b1 2<\/span><strong>\u039a\u03bb\u03b5\u03b9\u03b4\u03ce\u03c3\u03c4\u03b5 \u03c4\u03bf hardware topology<\/strong>\n<p>\u039a\u03b1\u03c4\u03b1\u03b3\u03c1\u03ac\u03c8\u03c4\u03b5 GPU \u03c4\u03cd\u03c0\u03bf, \u03b1\u03c1\u03b9\u03b8\u03bc\u03cc EP ranks, NVLink \u03b4\u03b9\u03b1\u03b4\u03c1\u03bf\u03bc\u03ad\u03c2, \u03b4\u03b9\u03b1\u03b8\u03ad\u03c3\u03b9\u03bc\u03b7 symmetric memory \u03ba\u03b1\u03b9 \u03ad\u03ba\u03b4\u03bf\u03c3\u03b7 \u03c4\u03bf\u03c5 framework \u03cc\u03c0\u03bf\u03c5 \u03b8\u03b1 \u03c4\u03c1\u03ad\u03be\u03b5\u03b9 \u03b7 \u03b4\u03bf\u03ba\u03b9\u03bc\u03ae.<\/p>\n<\/li>\n<li><span class=\"td-step-kicker\">\u0392\u03ae\u03bc\u03b1 3<\/span><strong>\u03a5\u03c0\u03bf\u03bb\u03bf\u03b3\u03af\u03c3\u03c4\u03b5 \u03c4\u03b1 prefetch slots B<\/strong>\n<p>\u0393\u03b9\u03b1 training \u03b5\u03bb\u03ad\u03b3\u03be\u03c4\u03b5 \u03c4\u03b7\u03bd \u03b1\u03c0\u03b1\u03af\u03c4\u03b7\u03c3\u03b7 B = E\/R. \u0393\u03b9\u03b1 inference \u03b4\u03bf\u03ba\u03b9\u03bc\u03ac\u03c3\u03c4\u03b5 \u03bc\u03b9\u03ba\u03c1\u03cc\u03c4\u03b5\u03c1\u03bf B \u03bc\u03cc\u03bd\u03bf \u03bc\u03b5 \u03bc\u03ad\u03c4\u03c1\u03b7\u03c3\u03b7 \u03c4\u03bf\u03c5 \u03ba\u03cc\u03c3\u03c4\u03bf\u03c5\u03c2 \u03b1\u03c0\u03bf\u03bc\u03b1\u03ba\u03c1\u03c5\u03c3\u03bc\u03ad\u03bd\u03b7\u03c2 \u03b1\u03bd\u03ac\u03b3\u03bd\u03c9\u03c3\u03b7\u03c2.<\/p>\n<\/li>\n<li><span class=\"td-step-kicker\">\u0392\u03ae\u03bc\u03b1 4<\/span><strong>\u0395\u03c0\u03b1\u03bb\u03b7\u03b8\u03b5\u03cd\u03c3\u03c4\u03b5 \u03c4\u03bf lifecycle \u03c4\u03c9\u03bd zero-copy views<\/strong>\n<p>\u0392\u03b5\u03b2\u03b1\u03b9\u03c9\u03b8\u03b5\u03af\u03c4\u03b5 \u03cc\u03c4\u03b9 \u03ba\u03b1\u03bd\u03ad\u03bd\u03b1 view \u03b4\u03b5\u03bd \u03ba\u03c1\u03b1\u03c4\u03b9\u03ad\u03c4\u03b1\u03b9 \u03bc\u03b5\u03c4\u03ac \u03c4\u03b7\u03bd \u03b5\u03c0\u03cc\u03bc\u03b5\u03bd\u03b7 dispatch \u03ae combine \u03ba\u03b1\u03b9 \u03cc\u03c4\u03b9 \u03c4\u03bf autograd \u03b4\u03b5\u03bd \u03c4\u03bf \u03b1\u03c0\u03bf\u03b8\u03b7\u03ba\u03b5\u03cd\u03b5\u03b9 \u03b3\u03b9\u03b1 backward.<\/p>\n<\/li>\n<li><span class=\"td-step-kicker\">\u0392\u03ae\u03bc\u03b1 5<\/span><strong>\u03a3\u03c5\u03b3\u03ba\u03c1\u03af\u03bd\u03b5\u03c4\u03b5 \u03bf\u03bb\u03cc\u03ba\u03bb\u03b7\u03c1\u03bf \u03c4\u03bf critical path<\/strong>\n<p>\u039c\u03b5\u03c4\u03c1\u03ae\u03c3\u03c4\u03b5 planning, weight prefetch, dispatch, combine, group GEMM, gradient reduction, peak \u03bc\u03bd\u03ae\u03bc\u03b7 \u03ba\u03b1\u03b9 OOM \u2014 \u03cc\u03c7\u03b9 \u03bc\u03cc\u03bd\u03bf \u03ad\u03bd\u03b1 \u03b5\u03c0\u03b9\u03ba\u03bf\u03b9\u03bd\u03c9\u03bd\u03b9\u03b1\u03ba\u03cc kernel.<\/p>\n<\/li>\n<li><span class=\"td-step-kicker\">\u0392\u03ae\u03bc\u03b1 6<\/span><strong>\u039f\u03c1\u03af\u03c3\u03c4\u03b5 acceptance criteria \u03ba\u03b1\u03b9 fallback<\/strong>\n<p>\u0391\u03c0\u03bf\u03c6\u03b1\u03c3\u03af\u03c3\u03c4\u03b5 \u03b5\u03ba \u03c4\u03c9\u03bd \u03c0\u03c1\u03bf\u03c4\u03ad\u03c1\u03c9\u03bd \u03c0\u03bf\u03b9\u03b1 \u03b2\u03b5\u03bb\u03c4\u03af\u03c9\u03c3\u03b7 \u03b4\u03b9\u03ba\u03b1\u03b9\u03bf\u03bb\u03bf\u03b3\u03b5\u03af \u03c4\u03b7\u03bd \u03b5\u03bd\u03c3\u03c9\u03bc\u03ac\u03c4\u03c9\u03c3\u03b7, \u03c0\u03bf\u03b9\u03bf test \u03b1\u03c0\u03bf\u03b4\u03b5\u03b9\u03ba\u03bd\u03cd\u03b5\u03b9 \u03bf\u03c1\u03b8\u03cc\u03c4\u03b7\u03c4\u03b1 \u03ba\u03b1\u03b9 \u03c0\u03ce\u03c2 \u03b5\u03c0\u03b9\u03c3\u03c4\u03c1\u03ad\u03c6\u03b5\u03b9 \u03c4\u03bf \u03c3\u03cd\u03c3\u03c4\u03b7\u03bc\u03b1 \u03c3\u03c4\u03b7\u03bd \u03c0\u03c1\u03bf\u03b7\u03b3\u03bf\u03cd\u03bc\u03b5\u03bd\u03b7 \u03c3\u03c4\u03bf\u03af\u03b2\u03b1.<\/p>\n<\/li>\n<\/ol>\n<\/div>\n<section class=\"td-service-cta\">\n<div class=\"td-service-cta-content\">\n<p class=\"td-service-cta-eyebrow\">\u0391\u03c5\u03c4\u03bf\u03bc\u03b1\u03c4\u03b9\u03c3\u03bc\u03bf\u03af \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03ae\u03c3\u03b5\u03c9\u03bd &amp; AI \u03b1\u03c0\u03cc \u03c4\u03b7\u03bd TWO DOTS<\/p>\n<p class=\"td-service-cta-title\">\u039c\u03b5\u03c4\u03b1\u03c4\u03c1\u03ad\u03c8\u03c4\u03b5 \u03c4\u03b9\u03c2 \u03c5\u03c0\u03bf\u03c3\u03c7\u03ad\u03c3\u03b5\u03b9\u03c2 \u03b5\u03bd\u03cc\u03c2 AI stack \u03c3\u03b5 \u03bc\u03b5\u03c4\u03c1\u03ae\u03c3\u03b9\u03bc\u03b1 \u03ba\u03c1\u03b9\u03c4\u03ae\u03c1\u03b9\u03b1 \u03b1\u03c0\u03bf\u03b4\u03bf\u03c7\u03ae\u03c2.<\/p>\n<p>\u0393\u03b9\u03b1 \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03b7\u03c3\u03b9\u03b1\u03ba\u03ac AI workflows, \u03b7 TWO DOTS \u03c7\u03b1\u03c1\u03c4\u03bf\u03b3\u03c1\u03b1\u03c6\u03b5\u03af \u03b1\u03c0\u03b1\u03b9\u03c4\u03ae\u03c3\u03b5\u03b9\u03c2, \u03b4\u03b5\u03b4\u03bf\u03bc\u03ad\u03bd\u03b1, \u03b5\u03bd\u03c3\u03c9\u03bc\u03b1\u03c4\u03ce\u03c3\u03b5\u03b9\u03c2, \u03b4\u03bf\u03ba\u03b9\u03bc\u03ad\u03c2, monitoring \u03ba\u03b1\u03b9 fallback, \u03ce\u03c3\u03c4\u03b5 \u03b7 \u03b5\u03c0\u03b9\u03bb\u03bf\u03b3\u03ae \u03c4\u03b5\u03c7\u03bd\u03bf\u03bb\u03bf\u03b3\u03af\u03b1\u03c2 \u03bd\u03b1 \u03c3\u03c4\u03b7\u03c1\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 \u03c3\u03b5 \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03ac \u03c3\u03b5\u03bd\u03ac\u03c1\u03b9\u03b1 \u03ba\u03b1\u03b9 \u03cc\u03c7\u03b9 \u03bc\u03cc\u03bd\u03bf \u03c3\u03b5 demo \u03ae benchmark \u03c4\u03c1\u03af\u03c4\u03bf\u03c5.<\/p>\n<div class=\"td-service-cta-actions\"><a class=\"td-service-cta-button\" href=\"https:\/\/twodots.gr\/aftomatismoi-epicheiriseon-ai\/\">\u0394\u03b5\u03af\u03c4\u03b5 \u03c4\u03bf\u03c5\u03c2 \u0391\u03c5\u03c4\u03bf\u03bc\u03b1\u03c4\u03b9\u03c3\u03bc\u03bf\u03cd\u03c2 \u0395\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03ae\u03c3\u03b5\u03c9\u03bd &amp; AI<\/a><\/div>\n<\/div>\n<\/section>\n<section id=\"sychnes-erotiseis\" class=\"td-faq-section\">\n<div class=\"td-faq\">\n<p class=\"td-faq-heading\">\u03a3\u03c5\u03c7\u03bd\u03ad\u03c2 \u03b5\u03c1\u03c9\u03c4\u03ae\u03c3\u03b5\u03b9\u03c2<\/p>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u03a4\u03b9 \u03b5\u03af\u03bd\u03b1\u03b9 \u03c4\u03bf MoonEP;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u03a4\u03bf MoonEP \u03b5\u03af\u03bd\u03b1\u03b9 open-source \u03b2\u03b9\u03b2\u03bb\u03b9\u03bf\u03b8\u03ae\u03ba\u03b7 \u03b5\u03c0\u03b9\u03ba\u03bf\u03b9\u03bd\u03c9\u03bd\u03af\u03b1\u03c2 expert parallelism \u03c4\u03b7\u03c2 Moonshot AI \u03b3\u03b9\u03b1 \u03ba\u03b1\u03c4\u03b1\u03bd\u03b5\u03bc\u03b7\u03bc\u03ad\u03bd\u03b1 Mixture-of-Experts workloads. \u03a3\u03c7\u03b5\u03b4\u03b9\u03ac\u03b6\u03b5\u03b9 \u03b4\u03c5\u03bd\u03b1\u03bc\u03b9\u03ba\u03ac redundant experts \u03ce\u03c3\u03c4\u03b5 \u03bd\u03b1 \u03ba\u03b1\u03c4\u03b1\u03bd\u03ad\u03bc\u03b5\u03b9 \u03c0\u03c1\u03bf\u03b2\u03bb\u03ad\u03c8\u03b9\u03bc\u03b1 \u03c4\u03b1 routed tokens \u03b1\u03bd\u03ac rank.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u03a0\u03bf\u03b9\u03bf \u03c0\u03c1\u03cc\u03b2\u03bb\u03b7\u03bc\u03b1 \u03c0\u03c1\u03bf\u03c3\u03c0\u03b1\u03b8\u03b5\u03af \u03bd\u03b1 \u03bb\u03cd\u03c3\u03b5\u03b9;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u03a3\u03c4\u03bf\u03c7\u03b5\u03cd\u03b5\u03b9 \u03c3\u03c4\u03bf routing skew: \u03bf\u03c1\u03b9\u03c3\u03bc\u03ad\u03bd\u03bf\u03b9 experts \u03bb\u03b1\u03bc\u03b2\u03ac\u03bd\u03bf\u03c5\u03bd \u03c0\u03b5\u03c1\u03b9\u03c3\u03c3\u03cc\u03c4\u03b5\u03c1\u03b1 tokens, \u03bf home rank \u03c5\u03c0\u03b5\u03c1\u03c6\u03bf\u03c1\u03c4\u03ce\u03bd\u03b5\u03c4\u03b1\u03b9 \u03ba\u03b1\u03b9 \u03bf\u03bb\u03cc\u03ba\u03bb\u03b7\u03c1\u03bf \u03c4\u03bf iteration \u03c0\u03b5\u03c1\u03b9\u03bc\u03ad\u03bd\u03b5\u03b9 \u03c4\u03bf \u03c0\u03b9\u03bf \u03b1\u03c1\u03b3\u03cc \u03c3\u03b7\u03bc\u03b5\u03af\u03bf \u03c4\u03b7\u03c2 \u03c3\u03c5\u03bb\u03bb\u03bf\u03b3\u03b9\u03ba\u03ae\u03c2 \u03bb\u03b5\u03b9\u03c4\u03bf\u03c5\u03c1\u03b3\u03af\u03b1\u03c2.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u03a4\u03b9 \u03c3\u03b7\u03bc\u03b1\u03af\u03bd\u03b5\u03b9 \u03cc\u03c4\u03b9 \u03ba\u03ac\u03b8\u03b5 rank \u03bb\u03b1\u03bc\u03b2\u03ac\u03bd\u03b5\u03b9 S \u00d7 K tokens;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u03a4\u03bf S \u03b5\u03af\u03bd\u03b1\u03b9 \u03c4\u03b1 input tokens \u03b1\u03bd\u03ac rank \u03ba\u03b1\u03b9 \u03c4\u03bf K \u03bf\u03b9 experts \u03c0\u03bf\u03c5 \u03b5\u03c0\u03b9\u03bb\u03ad\u03b3\u03bf\u03bd\u03c4\u03b1\u03b9 \u03b3\u03b9\u03b1 \u03ba\u03ac\u03b8\u03b5 token. \u039f\u03b9 \u03b4\u03b7\u03bc\u03b9\u03bf\u03c5\u03c1\u03b3\u03bf\u03af \u03b4\u03b7\u03bb\u03ce\u03bd\u03bf\u03c5\u03bd \u03cc\u03c4\u03b9 \u03bf \u03c3\u03c7\u03b5\u03b4\u03b9\u03b1\u03c3\u03bc\u03cc\u03c2 \u03c4\u03bf\u03c5 MoonEP \u03ba\u03c1\u03b1\u03c4\u03ac \u03c4\u03bf \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc \u03c6\u03bf\u03c1\u03c4\u03af\u03bf \u03c5\u03c0\u03bf\u03bb\u03bf\u03b3\u03b9\u03c3\u03bc\u03bf\u03cd \u03b1\u03ba\u03c1\u03b9\u03b2\u03ce\u03c2 \u03c3\u03c4\u03bf \u03b3\u03b9\u03bd\u03cc\u03bc\u03b5\u03bd\u03bf S \u00d7 K \u03b3\u03b9\u03b1 \u03ba\u03ac\u03b8\u03b5 rank.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u03a0\u03ce\u03c2 \u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03b5\u03af redundant experts \u03c7\u03c9\u03c1\u03af\u03c2 \u03bd\u03b1 \u03b1\u03bb\u03bb\u03ac\u03b6\u03b5\u03b9 \u03c4\u03bf \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03bf;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u039f online planner \u03b5\u03c0\u03b9\u03bb\u03ad\u03b3\u03b5\u03b9 \u03c0\u03c1\u03bf\u03c3\u03c9\u03c1\u03b9\u03bd\u03ac \u03b1\u03bd\u03c4\u03af\u03b3\u03c1\u03b1\u03c6\u03b1 \u03c4\u03c9\u03bd hot experts, \u03c0\u03c1\u03bf\u03c6\u03bf\u03c1\u03c4\u03ce\u03bd\u03b5\u03b9 \u03c4\u03b1 \u03b2\u03ac\u03c1\u03b7 \u03c4\u03bf\u03c5\u03c2 \u03c3\u03b5 \u03b4\u03b9\u03b1\u03b8\u03ad\u03c3\u03b9\u03bc\u03b1 slots \u03ac\u03bb\u03bb\u03c9\u03bd ranks \u03ba\u03b1\u03b9, \u03c3\u03c4\u03bf training, \u03b5\u03c0\u03b9\u03c3\u03c4\u03c1\u03ad\u03c6\u03b5\u03b9 \u03c4\u03bf\u03c5\u03c2 gradients \u03c3\u03c4\u03bf\u03c5\u03c2 home ranks \u03bc\u03ad\u03c3\u03c9 \u03be\u03b5\u03c7\u03c9\u03c1\u03b9\u03c3\u03c4\u03ce\u03bd reduce buffers.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u0395\u03af\u03bd\u03b1\u03b9 \u03b1\u03bd\u03b5\u03be\u03ac\u03c1\u03c4\u03b7\u03c4\u03b1 \u03b5\u03c0\u03b1\u03bb\u03b7\u03b8\u03b5\u03c5\u03bc\u03ad\u03bd\u03b1 \u03c4\u03b1 benchmarks \u03c4\u03bf\u03c5 MoonEP;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u038c\u03c7\u03b9 \u03b1\u03c0\u03cc \u03c4\u03b7\u03bd TWO DOTS. \u03a4\u03b1 \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03ad\u03c3\u03bc\u03b1\u03c4\u03b1 \u03ba\u03b1\u03b9 \u03c4\u03b1 \u03b3\u03c1\u03b1\u03c6\u03ae\u03bc\u03b1\u03c4\u03b1 \u03c0\u03c1\u03bf\u03ad\u03c1\u03c7\u03bf\u03bd\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc \u03c4\u03bf \u03b4\u03b7\u03bc\u03cc\u03c3\u03b9\u03bf \u03b1\u03c0\u03bf\u03b8\u03b5\u03c4\u03ae\u03c1\u03b9\u03bf \u03c4\u03c9\u03bd \u03b4\u03b7\u03bc\u03b9\u03bf\u03c5\u03c1\u03b3\u03ce\u03bd. \u03a7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03b1\u03bd\u03b1\u03c0\u03b1\u03c1\u03b1\u03b3\u03c9\u03b3\u03ae \u03c3\u03c4\u03bf \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc hardware, framework, \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03bf \u03ba\u03b1\u03b9 routing profile \u03c4\u03b7\u03c2 \u03bf\u03bc\u03ac\u03b4\u03b1\u03c2.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u039c\u03c0\u03bf\u03c1\u03b5\u03af \u03c4\u03bf MoonEP \u03bd\u03b1 \u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03b7\u03b8\u03b5\u03af \u03b3\u03b9\u03b1 inference;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u0397 \u03c4\u03b5\u03ba\u03bc\u03b7\u03c1\u03af\u03c9\u03c3\u03b7 \u03c0\u03b5\u03c1\u03b9\u03b3\u03c1\u03ac\u03c6\u03b5\u03b9 inference \u03c7\u03c9\u03c1\u03af\u03c2 gradient reduction \u03ba\u03b1\u03b9 \u03b5\u03c0\u03b9\u03c4\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bc\u03b9\u03ba\u03c1\u03cc\u03c4\u03b5\u03c1\u03bf B \u03b1\u03c0\u03cc \u03c4\u03bf E\/R, \u03c0\u03c1\u03bf\u03c4\u03b5\u03af\u03bd\u03bf\u03bd\u03c4\u03b1\u03c2 B \u03af\u03c3\u03bf \u03bc\u03b5 3 \u03ad\u03c9\u03c2 4. \u0391\u03bd \u03c4\u03b1 slots \u03b4\u03b5\u03bd \u03b5\u03c0\u03b1\u03c1\u03ba\u03bf\u03cd\u03bd, \u03b7 \u03b1\u03bd\u03ac\u03b3\u03bd\u03c9\u03c3\u03b7 \u03b3\u03af\u03bd\u03b5\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc \u03c4\u03bf\u03bd home rank \u03bc\u03b5 \u03c0\u03b9\u03b8\u03b1\u03bd\u03ae \u03b5\u03c0\u03b9\u03b2\u03c1\u03ac\u03b4\u03c5\u03bd\u03c3\u03b7.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u03a0\u03bf\u03b9\u03b5\u03c2 \u03c5\u03c0\u03bf\u03b4\u03bf\u03bc\u03ad\u03c2 \u03b1\u03c0\u03b1\u03b9\u03c4\u03b5\u03af \u03bc\u03b9\u03b1 \u03c3\u03bf\u03b2\u03b1\u03c1\u03ae \u03b4\u03bf\u03ba\u03b9\u03bc\u03ae;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u03a4\u03b1 \u03c0\u03b1\u03c1\u03b5\u03c7\u03cc\u03bc\u03b5\u03bd\u03b1 tests \u03b1\u03c0\u03b1\u03b9\u03c4\u03bf\u03cd\u03bd \u03c0\u03bf\u03bb\u03bb\u03b1\u03c0\u03bb\u03ad\u03c2 GPU \u03ba\u03b1\u03b9 NVLink. \u0397 \u03b5\u03bd\u03c3\u03c9\u03bc\u03ac\u03c4\u03c9\u03c3\u03b7 \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03c3\u03c5\u03bd\u03b5\u03c7\u03cc\u03bc\u03b5\u03bd\u03b1 symmetric-memory weight tensors, \u03c3\u03c9\u03c3\u03c4\u03ac prefetch \u03ba\u03b1\u03b9 reduce buffers, \u03c3\u03c5\u03bc\u03b2\u03b1\u03c4\u03cc group GEMM \u03ba\u03b1\u03b9 \u03b1\u03c3\u03c6\u03b1\u03bb\u03ad\u03c2 lifecycle \u03b3\u03b9\u03b1 \u03c4\u03b1 zero-copy views.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u03a0\u03cc\u03c4\u03b5 \u03ad\u03c7\u03b5\u03b9 \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03b7\u03bc\u03b1\u03c4\u03b9\u03ba\u03ae \u03b1\u03be\u03af\u03b1 \u03b7 \u03b1\u03be\u03b9\u03bf\u03bb\u03cc\u03b3\u03b7\u03c3\u03b7 \u03c4\u03bf\u03c5 MoonEP;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u038c\u03c4\u03b1\u03bd \u03bc\u03b9\u03b1 \u03bf\u03bc\u03ac\u03b4\u03b1 \u03b5\u03ba\u03c0\u03b1\u03b9\u03b4\u03b5\u03cd\u03b5\u03b9 \u03ae \u03c3\u03b5\u03c1\u03b2\u03af\u03c1\u03b5\u03b9 MoE \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03b1 \u03c3\u03b5 \u03b4\u03b9\u03ba\u03ae \u03c4\u03b7\u03c2 \u03c5\u03c0\u03bf\u03b4\u03bf\u03bc\u03ae, \u03ae \u03cc\u03c4\u03b1\u03bd \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03b5\u03bb\u03ad\u03b3\u03be\u03b5\u03b9 \u03c4\u03b5\u03c7\u03bd\u03b9\u03ba\u03bf\u03cd\u03c2 \u03b9\u03c3\u03c7\u03c5\u03c1\u03b9\u03c3\u03bc\u03bf\u03cd\u03c2 \u03c0\u03b1\u03c1\u03cc\u03c7\u03bf\u03c5 \u03b3\u03b9\u03b1 throughput, \u03bc\u03bd\u03ae\u03bc\u03b7 \u03ba\u03b1\u03b9 \u03ba\u03cc\u03c3\u03c4\u03bf\u03c2. \u0393\u03b9\u03b1 \u03b1\u03c0\u03bb\u03ae \u03c7\u03c1\u03ae\u03c3\u03b7 SaaS AI, \u03b7 \u03b5\u03c0\u03af\u03b4\u03c1\u03b1\u03c3\u03b7 \u03b5\u03af\u03bd\u03b1\u03b9 \u03c3\u03c5\u03bd\u03ae\u03b8\u03c9\u03c2 \u03ad\u03bc\u03bc\u03b5\u03c3\u03b7.<\/p>\n<\/div>\n<\/details>\n<\/div>\n<\/section>\n<div class=\"td-source-list\">\n<p id=\"piges\" class=\"td-source-list-title\">\u03a0\u03b7\u03b3\u03ad\u03c2<\/p>\n<ul>\n<li><a href=\"https:\/\/github.com\/MoonshotAI\/MoonEP\" target=\"_blank\" rel=\"noopener\">MoonshotAI: MoonEP repository \u03ba\u03b1\u03b9 \u03c4\u03b5\u03c7\u03bd\u03b9\u03ba\u03cc README<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/MoonshotAI\/MoonEP\/blob\/master\/benchmarks\/bench_vs_deepep.py\" target=\"_blank\" rel=\"noopener\">MoonshotAI: \u03b4\u03b7\u03bc\u03cc\u03c3\u03b9\u03bf benchmark MoonEP \u03ad\u03bd\u03b1\u03bd\u03c4\u03b9 DeepEP v2<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/MoonshotAI\/MoonEP\/blob\/master\/LICENSE\" target=\"_blank\" rel=\"noopener\">MoonshotAI: \u03ac\u03b4\u03b5\u03b9\u03b1 MIT \u03c4\u03bf\u03c5 MoonEP<\/a><\/li>\n<li><a href=\"https:\/\/github.com\/deepseek-ai\/DeepEP\" target=\"_blank\" rel=\"noopener\">DeepSeek: \u03b5\u03c0\u03af\u03c3\u03b7\u03bc\u03bf \u03b1\u03c0\u03bf\u03b8\u03b5\u03c4\u03ae\u03c1\u03b9\u03bf DeepEP<\/a><\/li>\n<li><a href=\"https:\/\/docs.nvidia.com\/cutlass\/latest\/media\/docs\/pythonDSL\/cute_dsl_api.html\" target=\"_blank\" rel=\"noopener\">NVIDIA: \u03b5\u03c0\u03af\u03c3\u03b7\u03bc\u03b7 \u03c4\u03b5\u03ba\u03bc\u03b7\u03c1\u03af\u03c9\u03c3\u03b7 CUTLASS CuTe DSL<\/a><\/li>\n<\/ul>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>\u03a4\u03bf MoonEP \u03b5\u03be\u03b9\u03c3\u03bf\u03c1\u03c1\u03bf\u03c0\u03b5\u03af \u03c4\u03b1 AI experts \u03c3\u03c4\u03b1 MoE \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03b1 \u03bc\u03b5 redundant experts, zero copy \u03ba\u03b1\u03b9 \u03c3\u03c4\u03b1\u03c4\u03b9\u03ba\u03ac shapes. \u0394\u03b5\u03af\u03c4\u03b5 benchmarks, \u03b1\u03c0\u03b1\u03b9\u03c4\u03ae\u03c3\u03b5\u03b9\u03c2 \u03ba\u03b1\u03b9 \u03ba\u03c1\u03b9\u03c4\u03ae\u03c1\u03b9\u03b1 \u03c5\u03b9\u03bf\u03b8\u03ad\u03c4\u03b7\u03c3\u03b7\u03c2.<\/p>","protected":false},"author":1,"featured_media":88098,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_gspb_post_css":"","content-type":"","footnotes":""},"categories":[199],"tags":[8988,7477,19035,19034,19036],"class_list":["post-87777","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-techniti-noimosyni","tag-ai-infrastructure","tag-machine-learning","tag-mixture-of-experts","tag-moonep","tag-moonshot-ai"],"blocksy_meta":[],"_links":{"self":[{"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/posts\/87777","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/comments?post=87777"}],"version-history":[{"count":0,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/posts\/87777\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/media\/88098"}],"wp:attachment":[{"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/media?parent=87777"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/categories?post=87777"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/tags?post=87777"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}