{"id":97814,"date":"2026-09-19T16:32:34","date_gmt":"2026-09-19T13:32:34","guid":{"rendered":"https:\/\/twodots.gr\/?p=97814"},"modified":"2026-09-19T16:32:36","modified_gmt":"2026-09-19T13:32:36","slug":"sparse-reward-ai-hints-critics-teachers","status":"publish","type":"post","link":"https:\/\/twodots.gr\/bg\/sparse-reward-ai-hints-critics-teachers\/","title":{"rendered":"\u038c\u03c4\u03b1\u03bd \u03c4\u03bf reward \u03c3\u03c9\u03c0\u03b1\u03af\u03bd\u03b5\u03b9: hints, critics \u03ba\u03b1\u03b9 teachers \u03c3\u03c4\u03b7\u03bd \u03b5\u03ba\u03c0\u03b1\u03af\u03b4\u03b5\u03c5\u03c3\u03b7 AI"},"content":{"rendered":"<div class=\"td-article-lede\">\n<p><strong>\u0391\u03c0\u03ac\u03bd\u03c4\u03b7\u03c3\u03b7 \u03c0\u03c1\u03ce\u03c4\u03b1:<\/strong> \u03c3\u03c4\u03bf sparse reward, \u03cc\u03c4\u03b1\u03bd \u03c3\u03c7\u03b5\u03b4\u03cc\u03bd \u03cc\u03bb\u03b1 \u03c4\u03b1 rollouts \u03c0\u03b1\u03af\u03c1\u03bd\u03bf\u03c5\u03bd \u03bc\u03b7\u03b4\u03b5\u03bd\u03b9\u03ba\u03cc reward, \u03c4\u03bf GRPO \u03b4\u03b5\u03bd \u03ad\u03c7\u03b5\u03b9 \u03b1\u03c1\u03ba\u03b5\u03c4\u03ad\u03c2 \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03ad\u03c2 \u03b3\u03b9\u03b1 \u03bd\u03b1 \u03b2\u03b5\u03bb\u03c4\u03b9\u03ce\u03c3\u03b5\u03b9 \u03c4\u03b7\u03bd policy. \u0397 \u03bc\u03b5\u03bb\u03ad\u03c4\u03b7 \u00abHints, Critics, and Teachers\u00bb \u03b4\u03b5\u03af\u03c7\u03bd\u03b5\u03b9 \u03cc\u03c4\u03b9 hints, annealed teacher feedback \u03ba\u03b1\u03b9 \u03c3\u03c9\u03c3\u03c4\u03ac \u03c0\u03b1\u03c1\u03b1\u03bc\u03b5\u03c4\u03c1\u03bf\u03c0\u03bf\u03b9\u03b7\u03bc\u03ad\u03bd\u03bf\u03b9 critics \u03bc\u03c0\u03bf\u03c1\u03bf\u03cd\u03bd \u03bd\u03b1 \u03b1\u03bd\u03bf\u03af\u03be\u03bf\u03c5\u03bd \u03c4\u03b7 \u03b4\u03b9\u03b1\u03b4\u03c1\u03bf\u03bc\u03ae \u03c0\u03c1\u03bf\u03c2 \u03c7\u03c1\u03ae\u03c3\u03b9\u03bc\u03bf learning signal \u2014 \u03b1\u03bb\u03bb\u03ac \u03bc\u03cc\u03bd\u03bf \u03cc\u03c4\u03b1\u03bd \u03c4\u03bf prior \u03c6\u03c4\u03ac\u03bd\u03b5\u03b9 \u03c0\u03c1\u03ac\u03b3\u03bc\u03b1\u03c4\u03b9 \u03c3\u03c4\u03b7\u03bd policy.<\/p>\n<p>\u03a4\u03bf \u03c0\u03b9\u03bf \u03b1\u03bd\u03b8\u03b5\u03ba\u03c4\u03b9\u03ba\u03cc \u03bc\u03ac\u03b8\u03b7\u03bc\u03b1 \u03b4\u03b5\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03c0\u03bf\u03b9\u03bf arm \u03b2\u03b3\u03ae\u03ba\u03b5 \u03c0\u03c1\u03ce\u03c4\u03bf. \u0395\u03af\u03bd\u03b1\u03b9 \u03cc\u03c4\u03b9 \u03ad\u03bd\u03b1\u03c2 \u03bb\u03b1\u03bd\u03b8\u03b1\u03c3\u03bc\u03ad\u03bd\u03bf\u03c2 scorer \u03ae \u03ad\u03bd\u03b1 \u03ba\u03b1\u03ba\u03bf\u03c3\u03c7\u03b5\u03b4\u03b9\u03b1\u03c3\u03bc\u03ad\u03bd\u03bf benchmark \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b1\u03bd\u03c4\u03b9\u03c3\u03c4\u03c1\u03ad\u03c8\u03b5\u03b9 \u03c4\u03b7\u03bd \u03ba\u03b1\u03c4\u03ac\u03c4\u03b1\u03be\u03b7 \u03c4\u03c9\u03bd \u03bc\u03b5\u03b8\u03cc\u03b4\u03c9\u03bd. \u0393\u03b9\u03b1 \u03bf\u03bc\u03ac\u03b4\u03b5\u03c2 \u03c0\u03bf\u03c5 \u03b5\u03ba\u03c0\u03b1\u03b9\u03b4\u03b5\u03cd\u03bf\u03c5\u03bd, \u03b1\u03b3\u03bf\u03c1\u03ac\u03b6\u03bf\u03c5\u03bd \u03ae \u03b1\u03be\u03b9\u03bf\u03bb\u03bf\u03b3\u03bf\u03cd\u03bd AI, \u03bf \u03bc\u03b7\u03c7\u03b1\u03bd\u03b9\u03c3\u03bc\u03cc\u03c2 \u03bc\u03ac\u03b8\u03b7\u03c3\u03b7\u03c2 \u03ba\u03b1\u03b9 \u03c4\u03bf measurement pipeline \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 \u03c4\u03bf\u03bd \u03af\u03b4\u03b9\u03bf \u03b2\u03b1\u03b8\u03bc\u03cc \u03b5\u03bb\u03ad\u03b3\u03c7\u03bf\u03c5.<\/p>\n<\/div>\n<div class=\"td-article-toc\">\n<div class=\"td-toc-title\">\u0421\u044a\u0434\u044a\u0440\u0436\u0430\u043d\u0438\u0435<\/div>\n<ul>\n<li><a href=\"#sparse-reward-grpo\">\u0393\u03b9\u03b1\u03c4\u03af \u03c4\u03bf GRPO \u00ab\u03c0\u03b5\u03b9\u03bd\u03ac\u00bb \u03cc\u03c4\u03b1\u03bd \u03cc\u03bb\u03b5\u03c2 \u03bf\u03b9 \u03b1\u03c0\u03b1\u03bd\u03c4\u03ae\u03c3\u03b5\u03b9\u03c2 \u03b5\u03af\u03bd\u03b1\u03b9 \u03bb\u03ac\u03b8\u03bf\u03c2<\/a><\/li>\n<li><a href=\"#scoring-trap\">\u0397 \u03c0\u03c1\u03ce\u03c4\u03b7 \u03c0\u03b1\u03b3\u03af\u03b4\u03b1 \u03ae\u03c4\u03b1\u03bd \u03c4\u03bf scoring, \u03cc\u03c7\u03b9 \u03c4\u03bf \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03bf<\/a><\/li>\n<li><a href=\"#eleven-arms\">\u0388\u03bd\u03c4\u03b5\u03ba\u03b1 arms, \u03c4\u03c1\u03b5\u03b9\u03c2 \u03c4\u03cd\u03c0\u03bf\u03b9 prior \u03ba\u03b1\u03b9 \u03ba\u03bf\u03b9\u03bd\u03cc training budget<\/a><\/li>\n<li><a href=\"#delivered-prior\">\u03a4\u03bf prior \u03b2\u03bf\u03b7\u03b8\u03ac \u03cc\u03c4\u03b1\u03bd \u03c0\u03b1\u03c1\u03b1\u03b4\u03af\u03b4\u03b5\u03c4\u03b1\u03b9 \u03c3\u03c4\u03b7\u03bd policy<\/a><\/li>\n<li><a href=\"#hints-exploration\">\u03a4\u03b1 hints \u03ba\u03ad\u03c1\u03b4\u03b9\u03c3\u03b1\u03bd \u03bc\u03ad\u03c3\u03c9 exploration<\/a><\/li>\n<li><a href=\"#critic-loss\">\u039c\u03b9\u03b1 \u03b1\u03bb\u03bb\u03b1\u03b3\u03ae \u03c3\u03c4\u03bf critic loss \u03ac\u03be\u03b9\u03b6\u03b5 14,4 \u03bc\u03bf\u03bd\u03ac\u03b4\u03b5\u03c2<\/a><\/li>\n<li><a href=\"#distillation-ceiling\">\u0397 distillation \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03c0\u03c1\u03cc\u03b3\u03c1\u03b1\u03bc\u03bc\u03b1 \u03ba\u03b1\u03b9 \u03ad\u03c7\u03b5\u03b9 ceiling<\/a><\/li>\n<li><a href=\"#benchmark-inversion\">\u03a4\u03bf benchmark \u03c0\u03bf\u03c5 \u03b1\u03bd\u03c4\u03ad\u03c3\u03c4\u03c1\u03b5\u03c6\u03b5 \u03c4\u03b7\u03bd \u03ba\u03b1\u03c4\u03ac\u03c4\u03b1\u03be\u03b7<\/a><\/li>\n<li><a href=\"#verification-audit\">Scoring audit, blind judging \u03ba\u03b1\u03b9 hard gates<\/a><\/li>\n<li><a href=\"#business-ai\">\u03a4\u03b9 \u03c3\u03b7\u03bc\u03b1\u03af\u03bd\u03b5\u03b9 \u03b3\u03b9\u03b1 \u03c0\u03c1\u03bf\u03ca\u03cc\u03bd\u03c4\u03b1, e-commerce \u03ba\u03b1\u03b9 marketing AI<\/a><\/li>\n<li><a href=\"#implementation-checklist\">\u0388\u03be\u03b9 \u03ad\u03bb\u03b5\u03b3\u03c7\u03bf\u03b9 \u03b3\u03b9\u03b1 \u03b4\u03cd\u03c3\u03ba\u03bf\u03bb\u03bf AI evaluation<\/a><\/li>\n<li><a href=\"#limitations-conclusion\">\u03a0\u03b5\u03c1\u03b9\u03bf\u03c1\u03b9\u03c3\u03bc\u03bf\u03af \u03ba\u03b1\u03b9 \u03c3\u03c9\u03c3\u03c4\u03cc \u03c3\u03c5\u03bc\u03c0\u03ad\u03c1\u03b1\u03c3\u03bc\u03b1<\/a><\/li>\n<\/ul>\n<\/div>\n<h2 id=\"sparse-reward-grpo\">\u0393\u03b9\u03b1\u03c4\u03af \u03c4\u03bf GRPO \u00ab\u03c0\u03b5\u03b9\u03bd\u03ac\u00bb \u03cc\u03c4\u03b1\u03bd \u03cc\u03bb\u03b5\u03c2 \u03bf\u03b9 \u03b1\u03c0\u03b1\u03bd\u03c4\u03ae\u03c3\u03b5\u03b9\u03c2 \u03b5\u03af\u03bd\u03b1\u03b9 \u03bb\u03ac\u03b8\u03bf\u03c2<\/h2>\n<p>\u03a4\u03bf Group Relative Policy Optimization \u03c3\u03c5\u03b3\u03ba\u03c1\u03af\u03bd\u03b5\u03b9 rollouts \u03b3\u03b9\u03b1 \u03c4\u03bf \u03af\u03b4\u03b9\u03bf prompt \u03ba\u03b1\u03b9 \u03c5\u03c0\u03bf\u03bb\u03bf\u03b3\u03af\u03b6\u03b5\u03b9 \u03c3\u03c7\u03b5\u03c4\u03b9\u03ba\u03cc \u03c0\u03bb\u03b5\u03bf\u03bd\u03ad\u03ba\u03c4\u03b7\u03bc\u03b1 \u03bc\u03ad\u03c3\u03b1 \u03c3\u03c4\u03b7\u03bd \u03bf\u03bc\u03ac\u03b4\u03b1. \u0391\u03bd \u03cc\u03bb\u03b1 \u03c4\u03b1 rollouts \u03b5\u03af\u03bd\u03b1\u03b9 \u03bb\u03ac\u03b8\u03bf\u03c2 \u03ba\u03b1\u03b9 \u03c0\u03b1\u03af\u03c1\u03bd\u03bf\u03c5\u03bd \u03c4\u03bf \u03af\u03b4\u03b9\u03bf reward, \u03b4\u03b5\u03bd \u03c5\u03c0\u03ac\u03c1\u03c7\u03b5\u03b9 \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03ac \u03c0\u03bf\u03c5 \u03bd\u03b1 \u03ba\u03b1\u03c4\u03b5\u03c5\u03b8\u03cd\u03bd\u03b5\u03b9 \u03c4\u03b7\u03bd policy. \u0397 \u03bf\u03bc\u03ac\u03b4\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 all-wrong \u03ba\u03b1\u03b9, \u03c3\u03c4\u03bf setup \u03c4\u03b7\u03c2 \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1\u03c2, \u03c3\u03c5\u03bd\u03b5\u03b9\u03c3\u03c6\u03ad\u03c1\u03b5\u03b9 \u03bc\u03b7\u03b4\u03b5\u03bd\u03b9\u03ba\u03cc gradient \u03b1\u03c0\u03cc \u03c4\u03bf task reward.<\/p>\n<p>\u039f\u03b9 \u03b5\u03c1\u03b5\u03c5\u03bd\u03b7\u03c4\u03ad\u03c2 \u03be\u03b5\u03ba\u03af\u03bd\u03b7\u03c3\u03b1\u03bd \u03b1\u03c0\u03cc 52.964 \u03c0\u03c1\u03bf\u03b2\u03bb\u03ae\u03bc\u03b1\u03c4\u03b1 \u03b5\u03bd\u03cc\u03c2 MathV360K-derived corpus, \u03ad\u03ba\u03b1\u03bd\u03b1\u03bd \u03bf\u03ba\u03c4\u03ce rollouts \u03b1\u03bd\u03ac \u03c0\u03c1\u03cc\u03b2\u03bb\u03b7\u03bc\u03b1 \u03ba\u03b1\u03b9 \u03c0\u03b1\u03c1\u03ae\u03b3\u03b1\u03b3\u03b1\u03bd \u03c0\u03b5\u03c1\u03af\u03c0\u03bf\u03c5 428.000 generations. \u039c\u03b5 semantic scoring \u03b5\u03c0\u03ad\u03bb\u03b5\u03be\u03b1\u03bd 21.630 \u03c0\u03c1\u03bf\u03b2\u03bb\u03ae\u03bc\u03b1\u03c4\u03b1 \u03c0\u03bf\u03c5 \u03c4\u03bf base model \u03ad\u03bb\u03c5\u03bd\u03b5 \u03c4\u03bf \u03c0\u03bf\u03bb\u03cd \u03bc\u03af\u03b1 \u03c6\u03bf\u03c1\u03ac \u03c3\u03c4\u03b9\u03c2 \u03bf\u03ba\u03c4\u03ce. \u039a\u03c1\u03ac\u03c4\u03b7\u03c3\u03b1\u03bd 800 \u03b3\u03b9\u03b1 \u03b1\u03be\u03b9\u03bf\u03bb\u03cc\u03b3\u03b7\u03c3\u03b7 \u03ba\u03b1\u03b9 \u03b5\u03ba\u03c0\u03b1\u03af\u03b4\u03b5\u03c5\u03c3\u03b1\u03bd \u03c3\u03c4\u03b1 \u03c5\u03c0\u03cc\u03bb\u03bf\u03b9\u03c0\u03b1 20.830.<\/p>\n<p>\u03a3\u03b5 \u03b1\u03c5\u03c4\u03cc \u03c4\u03bf \u03b4\u03cd\u03c3\u03ba\u03bf\u03bb\u03bf pool, \u03c4\u03bf Qwen2-VL-2B \u03b1\u03c0\u03b1\u03bd\u03c4\u03bf\u03cd\u03c3\u03b5 \u03c3\u03c9\u03c3\u03c4\u03ac \u03bc\u03cc\u03bb\u03b9\u03c2 \u03c3\u03c4\u03bf 3,6% \u03c4\u03c9\u03bd rollouts \u03ba\u03b1\u03b9 \u03c4\u03bf 85% \u03ad\u03c9\u03c2 97% \u03c4\u03c9\u03bd \u03b1\u03c1\u03c7\u03b9\u03ba\u03ce\u03bd GRPO groups \u03ae\u03c4\u03b1\u03bd \u03bf\u03bb\u03cc\u03ba\u03bb\u03b7\u03c1\u03b1 \u03bb\u03ac\u03b8\u03bf\u03c2. \u03a4\u03bf \u03c0\u03c1\u03cc\u03b2\u03bb\u03b7\u03bc\u03b1 \u03b4\u03b5\u03bd \u03ae\u03c4\u03b1\u03bd \u03b1\u03c0\u03bb\u03ce\u03c2 \u03cc\u03c4\u03b9 \u03b7 \u03b5\u03ba\u03c0\u03b1\u03af\u03b4\u03b5\u03c5\u03c3\u03b7 \u03c7\u03c1\u03b5\u03b9\u03b1\u03b6\u03cc\u03c4\u03b1\u03bd \u03c0\u03b5\u03c1\u03b9\u03c3\u03c3\u03cc\u03c4\u03b5\u03c1\u03b1 samples. \u03a3\u03c7\u03b5\u03b4\u03cc\u03bd \u03cc\u03bb\u03b1 \u03c4\u03b1 samples \u03b5\u03c0\u03ad\u03c3\u03c4\u03c1\u03b5\u03c6\u03b1\u03bd \u03c4\u03bf \u03af\u03b4\u03b9\u03bf \u03bc\u03b7\u03b4\u03b5\u03bd\u03b9\u03ba\u03cc \u03bc\u03ae\u03bd\u03c5\u03bc\u03b1, \u03ac\u03c1\u03b1 \u03c4\u03bf \u03b5\u03c0\u03b9\u03c0\u03bb\u03ad\u03bf\u03bd compute \u03b4\u03b5\u03bd \u03b4\u03b7\u03bc\u03b9\u03bf\u03c5\u03c1\u03b3\u03bf\u03cd\u03c3\u03b5 \u03b1\u03c5\u03c4\u03bf\u03bc\u03ac\u03c4\u03c9\u03c2 \u03c3\u03c5\u03b3\u03ba\u03c1\u03b9\u03c4\u03b9\u03ba\u03cc \u03c3\u03ae\u03bc\u03b1.<\/p>\n<div class=\"td-chart td-chart--metrics\">\n<div class=\"td-chart-head\">\n<p class=\"td-chart-title\">\u03a4\u03bf sparse-reward \u03ba\u03b1\u03b8\u03b5\u03c3\u03c4\u03ce\u03c2 \u03c4\u03b7\u03c2 \u03b5\u03bb\u03b5\u03b3\u03c7\u03cc\u03bc\u03b5\u03bd\u03b7\u03c2 \u03c3\u03cd\u03b3\u03ba\u03c1\u03b9\u03c3\u03b7\u03c2<\/p>\n<p class=\"td-chart-subtitle\">\u03a4\u03b1 \u03bc\u03b5\u03b3\u03ad\u03b8\u03b7 \u03b1\u03c6\u03bf\u03c1\u03bf\u03cd\u03bd \u03c4\u03bf \u03c3\u03c5\u03b3\u03ba\u03b5\u03ba\u03c1\u03b9\u03bc\u03ad\u03bd\u03bf visual-math experiment \u03ba\u03b1\u03b9 \u03b4\u03b5\u03bd \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03bf\u03cd\u03bd \u03b3\u03b5\u03bd\u03b9\u03ba\u03ac benchmarks \u03b3\u03b9\u03b1 \u03ba\u03ac\u03b8\u03b5 AI \u03c3\u03cd\u03c3\u03c4\u03b7\u03bc\u03b1.<\/p>\n<\/div>\n<div class=\"td-chart-body\">\n<div class=\"td-metric-grid\">\n<div class=\"td-metric-card\"><span class=\"td-metric-value\">20.830<\/span><span class=\"td-metric-label\">\u03b4\u03cd\u03c3\u03ba\u03bf\u03bb\u03b1 \u03c0\u03c1\u03bf\u03b2\u03bb\u03ae\u03bc\u03b1\u03c4\u03b1 \u03c3\u03c4\u03bf \u03ba\u03bf\u03b9\u03bd\u03cc training pool \u03c4\u03c9\u03bd \u03ad\u03bd\u03c4\u03b5\u03ba\u03b1 arms<\/span><\/div>\n<div class=\"td-metric-card\"><span class=\"td-metric-value\">3,6%<\/span><span class=\"td-metric-label\">\u03c3\u03c9\u03c3\u03c4\u03ac base-model rollouts \u03bc\u03ad\u03c3\u03b1 \u03c3\u03c4\u03bf \u03b5\u03c0\u03b9\u03bb\u03b5\u03b3\u03bc\u03ad\u03bd\u03bf sparse pool<\/span><\/div>\n<div class=\"td-metric-card\"><span class=\"td-metric-value\">85\u201397%<\/span><span class=\"td-metric-label\">all-wrong GRPO groups \u03c3\u03c4\u03b7\u03bd \u03b1\u03c1\u03c7\u03ae \u03c4\u03b7\u03c2 \u03b5\u03ba\u03c0\u03b1\u03af\u03b4\u03b5\u03c5\u03c3\u03b7\u03c2<\/span><\/div>\n<div class=\"td-metric-card\"><span class=\"td-metric-value\">+14,4<\/span><span class=\"td-metric-label\">\u03c0\u03bf\u03c3\u03bf\u03c3\u03c4\u03b9\u03b1\u03af\u03b5\u03c2 \u03bc\u03bf\u03bd\u03ac\u03b4\u03b5\u03c2 \u03c3\u03c4\u03bf sparse-800 \u03b1\u03c0\u03cc \u03c4\u03b7\u03bd \u03b1\u03bb\u03bb\u03b1\u03b3\u03ae critic loss<\/span><\/div>\n<\/div>\n<\/div>\n<\/div>\n<p>\u0397 \u03af\u03b4\u03b9\u03b1 \u03b4\u03bf\u03bc\u03b9\u03ba\u03ae \u03b4\u03c5\u03c3\u03ba\u03bf\u03bb\u03af\u03b1 \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b5\u03bc\u03c6\u03b1\u03bd\u03b9\u03c3\u03c4\u03b5\u03af \u03c3\u03b5 agents \u03bc\u03b5 \u03c3\u03c0\u03ac\u03bd\u03b9\u03b1 \u03b5\u03c0\u03b9\u03c4\u03c5\u03c7\u03af\u03b1, \u03c3\u03b5 \u03c0\u03bf\u03bb\u03c5\u03b2\u03ae\u03bc\u03b1\u03c4\u03b5\u03c2 \u03c1\u03bf\u03ad\u03c2 \u03ba\u03b1\u03b9 \u03c3\u03b5 \u03c0\u03b5\u03c1\u03b9\u03b2\u03ac\u03bb\u03bb\u03bf\u03bd\u03c4\u03b1 \u03cc\u03c0\u03bf\u03c5 \u03bc\u03cc\u03bd\u03bf \u03c4\u03bf \u03c4\u03b5\u03bb\u03b9\u03ba\u03cc outcome \u03b2\u03b1\u03b8\u03bc\u03bf\u03bb\u03bf\u03b3\u03b5\u03af\u03c4\u03b1\u03b9. \u0391\u03c5\u03c4\u03cc \u03b4\u03b5\u03bd \u03c3\u03b7\u03bc\u03b1\u03af\u03bd\u03b5\u03b9 \u03cc\u03c4\u03b9 \u03bf\u03b9 \u03b1\u03ba\u03c1\u03b9\u03b2\u03b5\u03af\u03c2 \u03bc\u03ad\u03b8\u03bf\u03b4\u03bf\u03b9 \u03c4\u03bf\u03c5 paper \u03bc\u03b5\u03c4\u03b1\u03c6\u03ad\u03c1\u03bf\u03bd\u03c4\u03b1\u03b9 \u03b1\u03c5\u03c4\u03bf\u03cd\u03c3\u03b9\u03b5\u03c2 \u03c3\u03b5 \u03ba\u03ac\u03b8\u03b5 \u03ba\u03bb\u03ac\u03b4\u03bf. \u03a3\u03b7\u03bc\u03b1\u03af\u03bd\u03b5\u03b9 \u03cc\u03c4\u03b9 \u03bc\u03b9\u03b1 \u03bf\u03bc\u03ac\u03b4\u03b1 \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03c0\u03c1\u03ce\u03c4\u03b1 \u03bd\u03b1 \u03bc\u03b5\u03c4\u03c1\u03ae\u03c3\u03b5\u03b9 \u03c0\u03cc\u03c3\u03bf \u03c3\u03c5\u03c7\u03bd\u03ac \u03c4\u03bf reward \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03bf\u03c0\u03bf\u03b9\u03b5\u03af \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03ac \u03c4\u03b9\u03c2 \u03c0\u03c1\u03bf\u03c3\u03c0\u03ac\u03b8\u03b5\u03b9\u03b5\u03c2.<\/p>\n<h2 id=\"scoring-trap\">\u0397 \u03c0\u03c1\u03ce\u03c4\u03b7 \u03c0\u03b1\u03b3\u03af\u03b4\u03b1 \u03ae\u03c4\u03b1\u03bd \u03c4\u03bf scoring, \u03cc\u03c7\u03b9 \u03c4\u03bf \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03bf<\/h2>\n<p>\u03a0\u03c1\u03b9\u03bd \u03b5\u03ba\u03c0\u03b1\u03b9\u03b4\u03b5\u03c5\u03c4\u03b5\u03af \u03bf\u03c0\u03bf\u03b9\u03bf\u03b4\u03ae\u03c0\u03bf\u03c4\u03b5 arm, \u03b7 \u03bf\u03bc\u03ac\u03b4\u03b1 \u03b2\u03c1\u03ae\u03ba\u03b5 \u03bc\u03b9\u03b1 \u03b1\u03c3\u03c4\u03bf\u03c7\u03af\u03b1 \u03c0\u03bf\u03c5 \u03b8\u03b1 \u03b5\u03af\u03c7\u03b5 \u03bc\u03bf\u03bb\u03cd\u03bd\u03b5\u03b9 \u03bf\u03bb\u03cc\u03ba\u03bb\u03b7\u03c1\u03bf \u03c4\u03bf \u03c0\u03b5\u03af\u03c1\u03b1\u03bc\u03b1. \u039f training scorer \u03b1\u03c0\u03b1\u03b9\u03c4\u03bf\u03cd\u03c3\u03b5 \u03b1\u03c0\u03ac\u03bd\u03c4\u03b7\u03c3\u03b7 \u03c3\u03c4\u03b7 \u03bc\u03bf\u03c1\u03c6\u03ae \u00abAnswer: X\u00bb \u03ba\u03b1\u03b9 \u03ad\u03b4\u03b9\u03bd\u03b5 \u03c3\u03c4\u03bf base model \u03b5\u03c0\u03af\u03b4\u03bf\u03c3\u03b7 3,5%. \u038c\u03c4\u03b1\u03bd \u03c4\u03b1 \u03af\u03b4\u03b9\u03b1 outputs \u03b1\u03be\u03b9\u03bf\u03bb\u03bf\u03b3\u03ae\u03b8\u03b7\u03ba\u03b1\u03bd \u03c3\u03b7\u03bc\u03b1\u03c3\u03b9\u03bf\u03bb\u03bf\u03b3\u03b9\u03ba\u03ac, \u03b7 \u03b5\u03c0\u03af\u03b4\u03bf\u03c3\u03b7 \u03ad\u03b3\u03b9\u03bd\u03b5 43,4%. \u039f \u03b1\u03c5\u03c3\u03c4\u03b7\u03c1\u03cc\u03c2 extractor \u03bc\u03b5\u03c4\u03c1\u03bf\u03cd\u03c3\u03b5 \u03ba\u03c5\u03c1\u03af\u03c9\u03c2 format compliance \u03ba\u03b1\u03b9 \u03cc\u03c7\u03b9 \u03bc\u03b1\u03b8\u03b7\u03bc\u03b1\u03c4\u03b9\u03ba\u03ae \u03b9\u03ba\u03b1\u03bd\u03cc\u03c4\u03b7\u03c4\u03b1.<\/p>\n<p>\u0391\u03bd \u03c4\u03bf \u03b4\u03cd\u03c3\u03ba\u03bf\u03bb\u03bf pool \u03b5\u03af\u03c7\u03b5 \u03b5\u03c0\u03b9\u03bb\u03b5\u03b3\u03b5\u03af \u03bc\u03b5 \u03c4\u03bf\u03bd \u03c0\u03c1\u03ce\u03c4\u03bf scorer, \u03c0\u03bf\u03bb\u03bb\u03ac \u00ab\u03b4\u03cd\u03c3\u03ba\u03bf\u03bb\u03b1\u00bb \u03c0\u03b1\u03c1\u03b1\u03b4\u03b5\u03af\u03b3\u03bc\u03b1\u03c4\u03b1 \u03b8\u03b1 \u03ae\u03c4\u03b1\u03bd \u03b1\u03c0\u03bb\u03ce\u03c2 formatting failures. \u0393\u03b9\u03b1 \u03b1\u03c5\u03c4\u03cc \u03b7 \u03b5\u03c0\u03b9\u03bb\u03bf\u03b3\u03ae \u03b4\u03b5\u03b4\u03bf\u03bc\u03ad\u03bd\u03c9\u03bd \u03ad\u03b3\u03b9\u03bd\u03b5 \u03bc\u03b5 semantic score, \u03b5\u03bd\u03ce \u03bf\u03b9 \u03c4\u03b5\u03bb\u03b9\u03ba\u03ad\u03c2 \u03b1\u03ba\u03c1\u03af\u03b2\u03b5\u03b9\u03b5\u03c2 \u03c0\u03c1\u03bf\u03ae\u03bb\u03b8\u03b1\u03bd \u03b1\u03c0\u03cc model-blind LLM judge \u03c0\u03bf\u03c5 \u03b4\u03b9\u03ac\u03b2\u03b1\u03b6\u03b5 \u03bf\u03bb\u03cc\u03ba\u03bb\u03b7\u03c1\u03bf \u03c4\u03bf output \u03c7\u03c9\u03c1\u03af\u03c2 \u03bd\u03b1 \u03b3\u03bd\u03c9\u03c1\u03af\u03b6\u03b5\u03b9 \u03c0\u03bf\u03b9\u03bf training arm \u03c4\u03bf \u03c0\u03b1\u03c1\u03ae\u03b3\u03b1\u03b3\u03b5.<\/p>\n<aside class=\"td-article-note\"><strong>Measurement \u03c0\u03c1\u03b9\u03bd \u03b1\u03c0\u03cc optimization:<\/strong> \u03cc\u03c4\u03b1\u03bd \u03ad\u03bd\u03b1 KPI \u03b5\u03be\u03b1\u03c1\u03c4\u03ac\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc \u03c3\u03c5\u03b3\u03ba\u03b5\u03ba\u03c1\u03b9\u03bc\u03ad\u03bd\u03bf output format, \u03b7 \u03bf\u03bc\u03ac\u03b4\u03b1 \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b2\u03b5\u03bb\u03c4\u03b9\u03c3\u03c4\u03bf\u03c0\u03bf\u03b9\u03b5\u03af \u03c3\u03c5\u03bc\u03bc\u03cc\u03c1\u03c6\u03c9\u03c3\u03b7 \u03c3\u03c4\u03b7 \u03c6\u03cc\u03c1\u03bc\u03b1 \u03ba\u03b1\u03b9 \u03bd\u03b1 \u03c4\u03b7\u03bd \u03c0\u03b1\u03c1\u03b5\u03c1\u03bc\u03b7\u03bd\u03b5\u03cd\u03b5\u03b9 \u03c9\u03c2 \u03ba\u03b1\u03bb\u03cd\u03c4\u03b5\u03c1\u03b7 \u03b5\u03c0\u03af\u03bb\u03c5\u03c3\u03b7. \u03a4\u03bf content score, \u03c4\u03bf format score \u03ba\u03b1\u03b9 \u03b7 \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03ae \u03bf\u03bb\u03bf\u03ba\u03bb\u03ae\u03c1\u03c9\u03c3\u03b7 \u03c4\u03bf\u03c5 \u03c3\u03c4\u03cc\u03c7\u03bf\u03c5 \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 \u03be\u03b5\u03c7\u03c9\u03c1\u03b9\u03c3\u03c4\u03bf\u03cd\u03c2 \u03b5\u03bb\u03ad\u03b3\u03c7\u03bf\u03c5\u03c2.<\/aside>\n<p>\u03a4\u03bf \u03af\u03b4\u03b9\u03bf \u03c1\u03af\u03c3\u03ba\u03bf \u03b5\u03bc\u03c6\u03b1\u03bd\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 \u03c3\u03b5 \u03b5\u03c4\u03b1\u03b9\u03c1\u03b9\u03ba\u03ac evaluations: \u03ad\u03bd\u03b1 bot \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b1\u03c0\u03bf\u03c4\u03c5\u03b3\u03c7\u03ac\u03bd\u03b5\u03b9 \u03c3\u03b5 regex \u03c0\u03bf\u03c5 \u03c8\u03ac\u03c7\u03bd\u03b5\u03b9 \u03c3\u03c5\u03b3\u03ba\u03b5\u03ba\u03c1\u03b9\u03bc\u03ad\u03bd\u03b7 \u03c6\u03c1\u03ac\u03c3\u03b7, \u03c0\u03b1\u03c1\u03cc\u03c4\u03b9 \u03b4\u03af\u03bd\u03b5\u03b9 \u03c3\u03c9\u03c3\u03c4\u03ae \u03b1\u03c0\u03ac\u03bd\u03c4\u03b7\u03c3\u03b7, \u03ae \u03bd\u03b1 \u03c0\u03b5\u03c1\u03bd\u03ac \u03b5\u03c0\u03b5\u03b9\u03b4\u03ae \u03bc\u03b9\u03bc\u03b5\u03af\u03c4\u03b1\u03b9 \u03c4\u03bf template \u03c7\u03c9\u03c1\u03af\u03c2 \u03bd\u03b1 \u03bf\u03bb\u03bf\u03ba\u03bb\u03b7\u03c1\u03ce\u03bd\u03b5\u03b9 \u03c4\u03b7 \u03b4\u03bf\u03c5\u03bb\u03b5\u03b9\u03ac. \u0397 \u03b1\u03bd\u03ac\u03bb\u03c5\u03c3\u03b7 \u03b3\u03b9\u03b1 \u03c4\u03bf <a href=\"https:\/\/twodots.gr\/dpo-preference-data-audit-llm\/\">audit \u03c4\u03c9\u03bd preference data \u03c0\u03c1\u03b9\u03bd \u03b1\u03c0\u03cc \u03b5\u03ba\u03c0\u03b1\u03af\u03b4\u03b5\u03c5\u03c3\u03b7 LLM<\/a> \u03b1\u03bd\u03b1\u03b4\u03b5\u03b9\u03ba\u03bd\u03cd\u03b5\u03b9 \u03c4\u03b7\u03bd \u03af\u03b4\u03b9\u03b1 \u03b1\u03bd\u03ac\u03b3\u03ba\u03b7: \u03c0\u03c1\u03ce\u03c4\u03b1 \u03b5\u03bb\u03ad\u03b3\u03c7\u03bf\u03c5\u03bc\u03b5 \u03c4\u03bf \u03c3\u03ae\u03bc\u03b1 \u03ba\u03b1\u03b9 \u03bc\u03b5\u03c4\u03ac \u03b2\u03b5\u03bb\u03c4\u03b9\u03c3\u03c4\u03bf\u03c0\u03bf\u03b9\u03bf\u03cd\u03bc\u03b5 \u03c0\u03ac\u03bd\u03c9 \u03c4\u03bf\u03c5.<\/p>\n<h2 id=\"eleven-arms\">\u0388\u03bd\u03c4\u03b5\u03ba\u03b1 arms, \u03c4\u03c1\u03b5\u03b9\u03c2 \u03c4\u03cd\u03c0\u03bf\u03b9 prior \u03ba\u03b1\u03b9 \u03ba\u03bf\u03b9\u03bd\u03cc training budget<\/h2>\n<p>\u0397 \u03bc\u03b5\u03bb\u03ad\u03c4\u03b7 \u03c3\u03c5\u03bd\u03ad\u03ba\u03c1\u03b9\u03bd\u03b5 \u03ad\u03bd\u03c4\u03b5\u03ba\u03b1 arms \u03c0\u03ac\u03bd\u03c9 \u03c3\u03c4\u03bf \u03af\u03b4\u03b9\u03bf pool, \u03b3\u03b9\u03b1 500 steps, \u03bc\u03b5 \u03af\u03b4\u03b9\u03bf batch size, learning rate \u03ba\u03b1\u03b9 seed. \u03a4\u03bf baseline \u03ae\u03c4\u03b1\u03bd plain GRPO. \u03a4\u03b1 text-prior arms \u03c0\u03c1\u03cc\u03c3\u03b8\u03b5\u03c4\u03b1\u03bd \u03c4\u03bc\u03ae\u03bc\u03b1 reference solution \u03c3\u03c4\u03b7\u03bd \u03b1\u03c1\u03c7\u03ae \u03c4\u03bf\u03c5 rollout: hintonly, uft001 \u03ba\u03b1\u03b9 uft002. \u03a4\u03b1 \u03b4\u03cd\u03bf UFT variants \u03c0\u03c1\u03cc\u03c3\u03b8\u03b5\u03c4\u03b1\u03bd auxiliary negative log-likelihood loss \u03c3\u03c4\u03b1 hint tokens.<\/p>\n<p>\u03a4\u03b1 distribution-prior arms \u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03bf\u03cd\u03c3\u03b1\u03bd Qwen2-VL-7B teacher. \u03a4\u03bf OPD \u03b2\u03b1\u03c3\u03b9\u03b6\u03cc\u03c4\u03b1\u03bd \u03b1\u03c0\u03bf\u03ba\u03bb\u03b5\u03b9\u03c3\u03c4\u03b9\u03ba\u03ac \u03c3\u03b5 on-policy distillation, \u03c4\u03bf ReOPD \u03c0\u03c1\u03cc\u03c3\u03b8\u03b5\u03c4\u03b5 teacher-trajectory prefix replay, \u03c4\u03bf SRPO \u03b5\u03bd\u03b5\u03c1\u03b3\u03bf\u03c0\u03bf\u03b9\u03bf\u03cd\u03c3\u03b5 distillation \u03bc\u03cc\u03bd\u03bf \u03c3\u03b5 failed rollouts \u03ba\u03b1\u03b9 \u03c4\u03bf SAF \u03b5\u03c6\u03ac\u03c1\u03bc\u03bf\u03b6\u03b5 globally \u03ad\u03bd\u03b1 bounded teacher signal \u03c0\u03bf\u03c5 \u03bc\u03b5\u03b9\u03c9\u03bd\u03cc\u03c4\u03b1\u03bd \u03c3\u03c4\u03b1\u03b4\u03b9\u03b1\u03ba\u03ac. \u03a4\u03bf hintsaf \u03c3\u03c5\u03bd\u03b4\u03cd\u03b1\u03b6\u03b5 hint \u03ba\u03b1\u03b9 SAF.<\/p>\n<p>\u03a3\u03c4\u03b7\u03bd \u03c4\u03c1\u03af\u03c4\u03b7 \u03bf\u03b9\u03ba\u03bf\u03b3\u03ad\u03bd\u03b5\u03b9\u03b1, \u03c4\u03b1 VAPO \u03ba\u03b1\u03b9 VAPO-HL \u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b1\u03bd pretrained critic. \u03a4\u03bf VAPO \u03b5\u03af\u03c7\u03b5 clipped-MSE value loss, \u03b5\u03bd\u03ce \u03c4\u03bf VAPO-HL \u03b1\u03bd\u03c4\u03b9\u03ba\u03b1\u03c4\u03ad\u03c3\u03c4\u03b7\u03c3\u03b5 \u03b1\u03ba\u03c1\u03b9\u03b2\u03ce\u03c2 \u03b1\u03c5\u03c4\u03cc \u03c4\u03bf \u03c3\u03c4\u03bf\u03b9\u03c7\u03b5\u03af\u03bf \u03bc\u03b5 categorical HL-Gauss loss. \u0397 \u03ba\u03bf\u03b9\u03bd\u03ae \u03b2\u03ac\u03c3\u03b7 \u03b5\u03c0\u03b9\u03c4\u03c1\u03ad\u03c0\u03b5\u03b9 \u03c0\u03b9\u03bf \u03ba\u03b1\u03b8\u03b1\u03c1\u03ae \u03b1\u03c0\u03cc\u03b4\u03bf\u03c3\u03b7 \u03c4\u03c9\u03bd \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03ce\u03bd \u03c3\u03c4\u03bf\u03bd \u03bc\u03b7\u03c7\u03b1\u03bd\u03b9\u03c3\u03bc\u03cc \u03c4\u03bf\u03c5 prior, \u03c7\u03c9\u03c1\u03af\u03c2 \u03bd\u03b1 \u03ba\u03b1\u03c4\u03b1\u03c1\u03b3\u03b5\u03af \u03c4\u03bf\u03bd \u03c0\u03b5\u03c1\u03b9\u03bf\u03c1\u03b9\u03c3\u03bc\u03cc \u03cc\u03c4\u03b9 \u03cc\u03bb\u03b1 \u03c4\u03b1 arms \u03b5\u03ba\u03c0\u03b1\u03b9\u03b4\u03b5\u03cd\u03c4\u03b7\u03ba\u03b1\u03bd \u03bc\u03b5 \u03ad\u03bd\u03b1\u03bd \u03bc\u03cc\u03bd\u03bf seed.<\/p>\n<div class=\"td-comparison td-comparison-cards td-comparison-cards--horizontal\">\n<div class=\"td-comparison-grid td-comparison-grid--three\">\n<div class=\"td-platform-card\">\n<p class=\"td-comparison-title\">Text prior<\/p>\n<p>\u03a4\u03bc\u03ae\u03bc\u03b1 reference solution \u03bc\u03b5\u03c4\u03b1\u03c6\u03ad\u03c1\u03b5\u03b9 \u03c4\u03bf rollout \u03ba\u03bf\u03bd\u03c4\u03ac \u03c3\u03b5 \u03c0\u03b5\u03c1\u03b9\u03bf\u03c7\u03ae \u03cc\u03c0\u03bf\u03c5 \u03b7 \u03b5\u03c0\u03b9\u03c4\u03c5\u03c7\u03af\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 \u03b5\u03c6\u03b9\u03ba\u03c4\u03ae \u03ba\u03b1\u03b9 \u03b5\u03bc\u03c6\u03b1\u03bd\u03af\u03b6\u03b5\u03c4\u03b1\u03b9 \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03bf\u03c0\u03bf\u03b9\u03b7\u03bc\u03ad\u03bd\u03bf reward.<\/p>\n<div class=\"td-badge-row\"><span class=\"td-badge\">Hints<\/span><span class=\"td-badge\">Exploration<\/span><\/div>\n<\/div>\n<div class=\"td-platform-card td-platform-card--navy\">\n<p class=\"td-comparison-title\">Distribution prior<\/p>\n<p>\u0388\u03bd\u03b1\u03c2 \u03b9\u03c3\u03c7\u03c5\u03c1\u03cc\u03c4\u03b5\u03c1\u03bf\u03c2 teacher \u03b4\u03af\u03bd\u03b5\u03b9 \u03c0\u03c5\u03ba\u03bd\u03cc\u03c4\u03b5\u03c1\u03bf per-token \u03c3\u03ae\u03bc\u03b1, \u03bc\u03b5 \u03c4\u03bf \u03b1\u03c0\u03bf\u03c4\u03ad\u03bb\u03b5\u03c3\u03bc\u03b1 \u03bd\u03b1 \u03b5\u03be\u03b1\u03c1\u03c4\u03ac\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc gating, schedule \u03ba\u03b1\u03b9 teacher ceiling.<\/p>\n<div class=\"td-badge-row\"><span class=\"td-badge\">Distillation<\/span><span class=\"td-badge\">Annealing<\/span><\/div>\n<\/div>\n<div class=\"td-platform-card\">\n<p class=\"td-comparison-title\">Value prior<\/p>\n<p>\u039f critic \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03b5\u03af \u03bd\u03b1 \u03b4\u03ce\u03c3\u03b5\u03b9 token-level credit, \u03b1\u03bb\u03bb\u03ac \u03b7 \u03c0\u03b1\u03c1\u03ac\u03b4\u03bf\u03c3\u03b7 \u03c4\u03bf\u03c5 \u03c3\u03ae\u03bc\u03b1\u03c4\u03bf\u03c2 \u03b5\u03be\u03b1\u03c1\u03c4\u03ac\u03c4\u03b1\u03b9 \u03ba\u03c1\u03af\u03c3\u03b9\u03bc\u03b1 \u03b1\u03c0\u03cc \u03c4\u03bf\u03bd \u03c4\u03c1\u03cc\u03c0\u03bf \u03b5\u03ba\u03c0\u03b1\u03af\u03b4\u03b5\u03c5\u03c3\u03b7\u03c2 \u03c4\u03bf\u03c5 value head.<\/p>\n<div class=\"td-badge-row\"><span class=\"td-badge\">Critic<\/span><span class=\"td-badge\">HL-Gauss<\/span><\/div>\n<\/div>\n<\/div>\n<\/div>\n<h2 id=\"delivered-prior\">\u03a4\u03bf prior \u03b2\u03bf\u03b7\u03b8\u03ac \u03cc\u03c4\u03b1\u03bd \u03c0\u03b1\u03c1\u03b1\u03b4\u03af\u03b4\u03b5\u03c4\u03b1\u03b9 \u03c3\u03c4\u03b7\u03bd policy<\/h2>\n<p>\u03a3\u03c4\u03bf cross-domain DynaMath, \u03c4\u03b1 \u03ad\u03be\u03b9 arms \u03cc\u03c0\u03bf\u03c5 \u03c4\u03bf prior \u03ad\u03c6\u03c4\u03b1\u03bd\u03b5 \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03b5\u03c3\u03bc\u03b1\u03c4\u03b9\u03ba\u03ac \u03c3\u03c4\u03b7\u03bd policy \u03b2\u03c1\u03ad\u03b8\u03b7\u03ba\u03b1\u03bd \u03bc\u03b5\u03c4\u03b1\u03be\u03cd 30,6% \u03ba\u03b1\u03b9 32,9%. \u03a4\u03b1 \u03c5\u03c0\u03cc\u03bb\u03bf\u03b9\u03c0\u03b1 \u03c0\u03ad\u03bd\u03c4\u03b5 \u2014 \u03c4\u03bf baseline \u03ba\u03b1\u03b9 \u03bc\u03b7\u03c7\u03b1\u03bd\u03b9\u03c3\u03bc\u03bf\u03af \u03cc\u03c0\u03bf\u03c5 \u03c4\u03bf prior \u03c0\u03b5\u03c1\u03b9\u03bf\u03c1\u03b9\u03b6\u03cc\u03c4\u03b1\u03bd \u03b1\u03c0\u03cc \u03c4\u03bf\u03bd teacher, \u03b5\u03bd\u03b5\u03c1\u03b3\u03bf\u03c0\u03bf\u03b9\u03bf\u03cd\u03bd\u03c4\u03b1\u03bd \u03bc\u03b5 \u03c0\u03c1\u03bf\u03b2\u03bb\u03b7\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc gate \u03ae \u03c7\u03b1\u03bd\u03cc\u03c4\u03b1\u03bd \u03c3\u03b5 \u03ba\u03b1\u03ba\u03ac \u03c0\u03b1\u03c1\u03b1\u03bc\u03b5\u03c4\u03c1\u03bf\u03c0\u03bf\u03b9\u03b7\u03bc\u03ad\u03bd\u03bf critic \u2014 \u03b2\u03c1\u03ad\u03b8\u03b7\u03ba\u03b1\u03bd \u03bc\u03b5\u03c4\u03b1\u03be\u03cd 27,0% \u03ba\u03b1\u03b9 29,1%. \u039f\u03b9 \u03b4\u03cd\u03bf \u03c0\u03b5\u03c1\u03b9\u03bf\u03c7\u03ad\u03c2 \u03b4\u03b5\u03bd \u03b5\u03c0\u03b9\u03ba\u03b1\u03bb\u03cd\u03c0\u03c4\u03bf\u03bd\u03c4\u03b1\u03bd.<\/p>\n<p>\u03a4\u03bf uft002 \u03b5\u03af\u03c7\u03b5 32,9%, \u03c4\u03bf VAPO-HL 31,9% \u03ba\u03b1\u03b9 \u03c4\u03bf hintonly 31,2%, \u03b5\u03bd\u03ce \u03c4\u03bf plain GRPO \u03ad\u03c6\u03c4\u03b1\u03c3\u03b5 27,5%. \u039f\u03b9 \u03c3\u03c5\u03b3\u03b3\u03c1\u03b1\u03c6\u03b5\u03af\u03c2 \u03c0\u03c1\u03bf\u03b5\u03b9\u03b4\u03bf\u03c0\u03bf\u03b9\u03bf\u03cd\u03bd \u03cc\u03c4\u03b9 \u03b7 \u03c3\u03b5\u03b9\u03c1\u03ac \u03bc\u03ad\u03c3\u03b1 \u03c3\u03c4\u03b7\u03bd \u03b1\u03bd\u03ce\u03c4\u03b5\u03c1\u03b7 \u03bf\u03bc\u03ac\u03b4\u03b1 \u03b4\u03b5\u03bd \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03b4\u03b9\u03b1\u03b2\u03b1\u03c3\u03c4\u03b5\u03af \u03c9\u03c2 \u03c3\u03c4\u03b1\u03b8\u03b5\u03c1\u03cc ranking: \u03c4\u03bf \u03b5\u03cd\u03c1\u03bf\u03c2 \u03b5\u03af\u03bd\u03b1\u03b9 \u03bc\u03cc\u03bb\u03b9\u03c2 2,3 \u03bc\u03bf\u03bd\u03ac\u03b4\u03b5\u03c2, \u03c4\u03bf DynaMath \u03ad\u03c7\u03b5\u03b9 477 items \u03ba\u03b1\u03b9 \u03c5\u03c0\u03ac\u03c1\u03c7\u03b5\u03b9 \u03ad\u03bd\u03b1\u03c2 training seed.<\/p>\n<p>\u03a4\u03bf \u03b9\u03c3\u03c7\u03c5\u03c1\u03cc\u03c4\u03b5\u03c1\u03bf claim \u03b5\u03af\u03bd\u03b1\u03b9 \u03b5\u03c0\u03bf\u03bc\u03ad\u03bd\u03c9\u03c2 \u03bf \u03b4\u03b9\u03b1\u03c7\u03c9\u03c1\u03b9\u03c3\u03bc\u03cc\u03c2 delivered \u03ba\u03b1\u03b9 undelivered priors, \u03cc\u03c7\u03b9 \u03bf \u00ab\u03bd\u03b9\u03ba\u03b7\u03c4\u03ae\u03c2\u00bb. \u0393\u03b9\u03b1 ML teams \u03b1\u03c5\u03c4\u03cc \u03bc\u03b5\u03c4\u03b1\u03c6\u03c1\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03c3\u03b5 observability \u03bc\u03ad\u03c3\u03b1 \u03c3\u03c4\u03bf training loop. \u0394\u03b5\u03bd \u03b1\u03c1\u03ba\u03b5\u03af \u03bd\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 \u03b5\u03bd\u03b5\u03c1\u03b3\u03cc \u03ad\u03bd\u03b1 teacher loss, \u03ad\u03bd\u03b1 memory module \u03ae \u03ad\u03bd\u03b1\u03c2 verifier. \u03a7\u03c1\u03b5\u03b9\u03ac\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 \u03bc\u03b5\u03c4\u03c1\u03ae\u03c3\u03b5\u03b9\u03c2 \u03b3\u03b9\u03b1 \u03c4\u03bf \u03c0\u03cc\u03c4\u03b5 \u03b5\u03bd\u03b5\u03c1\u03b3\u03bf\u03c0\u03bf\u03b9\u03b5\u03af\u03c4\u03b1\u03b9, \u03c3\u03b5 \u03c0\u03bf\u03b9\u03b1 samples, \u03bc\u03b5 \u03c0\u03bf\u03b9\u03bf gradient \u03ba\u03b1\u03b9 \u03b1\u03bd \u03b7 \u03b5\u03c0\u03af\u03b4\u03c1\u03b1\u03c3\u03ae \u03c4\u03bf\u03c5 \u03b5\u03c0\u03b9\u03b2\u03b9\u03ce\u03bd\u03b5\u03b9 \u03bc\u03ad\u03c7\u03c1\u03b9 \u03c4\u03bf \u03c4\u03b5\u03bb\u03b9\u03ba\u03cc behavior.<\/p>\n<p>\u0397 \u03b1\u03c1\u03c7\u03ae \u03c3\u03c5\u03bd\u03b4\u03ad\u03b5\u03c4\u03b1\u03b9 \u03bc\u03b5 \u03c4\u03b7\u03bd <a href=\"https:\/\/twodots.gr\/epalithefsi-politikon-reinforcement-learning-ai-agent\/\">\u03b5\u03c0\u03b1\u03bb\u03ae\u03b8\u03b5\u03c5\u03c3\u03b7 \u03c0\u03bf\u03bb\u03b9\u03c4\u03b9\u03ba\u03ce\u03bd reinforcement learning<\/a>: \u03ad\u03bd\u03b1 \u03b5\u03bd\u03b5\u03c1\u03b3\u03bf\u03c0\u03bf\u03b9\u03b7\u03bc\u03ad\u03bd\u03bf control \u03b4\u03b5\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03b1\u03c0\u03cc\u03b4\u03b5\u03b9\u03be\u03b7 \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03b5\u03c3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc\u03c4\u03b7\u03c4\u03b1\u03c2. \u03a7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 observable \u03b5\u03c0\u03af\u03b4\u03c1\u03b1\u03c3\u03b7 \u03ba\u03b1\u03b9 \u03b1\u03bd\u03b5\u03be\u03ac\u03c1\u03c4\u03b7\u03c4\u03b7 \u03b1\u03be\u03b9\u03bf\u03bb\u03cc\u03b3\u03b7\u03c3\u03b7 \u03c4\u03b7\u03c2 policy \u03c0\u03bf\u03c5 \u03c0\u03c1\u03bf\u03ba\u03cd\u03c0\u03c4\u03b5\u03b9.<\/p>\n<h2 id=\"hints-exploration\">\u03a4\u03b1 hints \u03ba\u03ad\u03c1\u03b4\u03b9\u03c3\u03b1\u03bd \u03bc\u03ad\u03c3\u03c9 exploration<\/h2>\n<p>\u03a0\u03c1\u03b9\u03bd \u03b1\u03c0\u03cc \u03c4\u03bf training, \u03b7 \u03c0\u03b1\u03c1\u03bf\u03c7\u03ae \u03c4\u03bf\u03c5 \u03bc\u03b9\u03c3\u03bf\u03cd reference solution \u03b1\u03cd\u03be\u03b7\u03c3\u03b5 \u03c4\u03b7\u03bd \u03b1\u03ba\u03c1\u03af\u03b2\u03b5\u03b9\u03b1 \u03c4\u03bf\u03c5 base model \u03b1\u03c0\u03cc 7,2% \u03c3\u03b5 34,4%. \u03a4\u03bf precondition \u03ad\u03b4\u03b5\u03b9\u03be\u03b5 \u03cc\u03c4\u03b9 \u03c4\u03b1 hints \u03bc\u03c0\u03bf\u03c1\u03bf\u03cd\u03c3\u03b1\u03bd \u03cc\u03bd\u03c4\u03c9\u03c2 \u03bd\u03b1 \u03be\u03b5\u03ba\u03bb\u03b5\u03b9\u03b4\u03ce\u03c3\u03bf\u03c5\u03bd \u03c4\u03bf \u03b5\u03c0\u03b9\u03bb\u03b5\u03b3\u03bc\u03ad\u03bd\u03bf sparse pool. \u03a7\u03c9\u03c1\u03af\u03c2 \u03b1\u03c5\u03c4\u03cc\u03bd \u03c4\u03bf\u03bd \u03ad\u03bb\u03b5\u03b3\u03c7\u03bf, \u03b7 \u03bf\u03bc\u03ac\u03b4\u03b1 \u03b8\u03b1 \u03bc\u03c0\u03bf\u03c1\u03bf\u03cd\u03c3\u03b5 \u03bd\u03b1 \u03b5\u03c0\u03b5\u03bd\u03b4\u03cd\u03c3\u03b5\u03b9 \u03c3\u03b5 scaffold \u03c0\u03bf\u03c5 \u03b4\u03b5\u03bd \u03ac\u03bb\u03bb\u03b1\u03b6\u03b5 \u03c4\u03b7\u03bd \u03c0\u03b9\u03b8\u03b1\u03bd\u03cc\u03c4\u03b7\u03c4\u03b1 \u03b5\u03c0\u03b9\u03c4\u03c5\u03c7\u03af\u03b1\u03c2.<\/p>\n<p>\u03a3\u03c4\u03b1 \u03c0\u03b5\u03b9\u03c1\u03ac\u03bc\u03b1\u03c4\u03b1, \u03c4\u03bf hintonly \u03ae\u03c4\u03b1\u03bd \u03c3\u03c4\u03b1\u03c4\u03b9\u03c3\u03c4\u03b9\u03ba\u03ac \u03b1\u03b4\u03b9\u03b1\u03c7\u03ce\u03c1\u03b9\u03c3\u03c4\u03bf \u03b1\u03c0\u03cc \u03c4\u03b1 uft001 \u03ba\u03b1\u03b9 uft002 \u03c3\u03c4\u03bf sparse-800: 51,4%, 52,0% \u03ba\u03b1\u03b9 52,4%. \u03a4\u03bf \u03b2\u03b1\u03c3\u03b9\u03ba\u03cc \u03cc\u03c6\u03b5\u03bb\u03bf\u03c2 \u03c0\u03c1\u03bf\u03b5\u03c1\u03c7\u03cc\u03c4\u03b1\u03bd \u03b1\u03c0\u03cc \u03c4\u03b7\u03bd \u03ba\u03b1\u03b8\u03bf\u03b4\u03b7\u03b3\u03bf\u03cd\u03bc\u03b5\u03bd\u03b7 exploration \u03ba\u03b1\u03b9 \u03cc\u03c7\u03b9 \u03b1\u03c0\u03cc \u03c4\u03bf \u03c0\u03c1\u03cc\u03c3\u03b8\u03b5\u03c4\u03bf likelihood term \u03c0\u03ac\u03bd\u03c9 \u03c3\u03c4\u03b1 hint tokens. \u03a4\u03bf hint \u03bc\u03b5\u03c4\u03ad\u03c6\u03b5\u03c1\u03b5 \u03c4\u03bf rollout \u03c3\u03b5 \u03c0\u03b5\u03c1\u03b9\u03bf\u03c7\u03ae \u03cc\u03c0\u03bf\u03c5 \u03c5\u03c0\u03ae\u03c1\u03c7\u03b5 \u03c0\u03b9\u03b8\u03b1\u03bd\u03cc\u03c4\u03b7\u03c4\u03b1 \u03c3\u03c9\u03c3\u03c4\u03ae\u03c2 \u03bb\u03cd\u03c3\u03b7\u03c2 \u03ba\u03b1\u03b9 \u03ac\u03c1\u03b1 \u03c7\u03c1\u03ae\u03c3\u03b9\u03bc\u03bf reward.<\/p>\n<p>\u0397 \u03b1\u03bd\u03b1\u03bb\u03bf\u03b3\u03af\u03b1 hinted rollouts \u03bc\u03b5\u03b9\u03c9\u03bd\u03cc\u03c4\u03b1\u03bd \u03b1\u03c0\u03cc 95% \u03c3\u03b5 5% \u03c3\u03c4\u03b1 \u03c0\u03c1\u03ce\u03c4\u03b1 300 steps \u03ba\u03b1\u03b9 \u03bc\u03b5\u03c4\u03ac \u03bc\u03b7\u03b4\u03b5\u03bd\u03b9\u03b6\u03cc\u03c4\u03b1\u03bd. \u0397 policy \u03ad\u03c0\u03c1\u03b5\u03c0\u03b5 \u03c3\u03c4\u03b1\u03b4\u03b9\u03b1\u03ba\u03ac \u03bd\u03b1 \u03bf\u03bb\u03bf\u03ba\u03bb\u03b7\u03c1\u03ce\u03bd\u03b5\u03b9 \u03bc\u03cc\u03bd\u03b7 \u03c4\u03b7\u03c2 \u03c4\u03b7 \u03b4\u03b9\u03b1\u03b4\u03c1\u03bf\u03bc\u03ae. \u0391\u03c5\u03c4\u03cc \u03b4\u03b5\u03bd \u03b1\u03c0\u03bf\u03b4\u03b5\u03b9\u03ba\u03bd\u03cd\u03b5\u03b9 \u03cc\u03c4\u03b9 \u03ba\u03ac\u03b8\u03b5 hint \u03b5\u03af\u03bd\u03b1\u03b9 \u03b1\u03c3\u03c6\u03b1\u03bb\u03ad\u03c2: \u03c4\u03b1 hint arms \u03b4\u03b5\u03bd \u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b1\u03bd importance-weighted \u03b4\u03b9\u03cc\u03c1\u03b8\u03c9\u03c3\u03b7 \u03b3\u03b9\u03b1 \u03c4\u03bf off-policy \u03c3\u03c4\u03bf\u03b9\u03c7\u03b5\u03af\u03bf, \u03ba\u03ac\u03c4\u03b9 \u03c0\u03bf\u03c5 \u03bf\u03b9 \u03af\u03b4\u03b9\u03bf\u03b9 \u03bf\u03b9 \u03c3\u03c5\u03b3\u03b3\u03c1\u03b1\u03c6\u03b5\u03af\u03c2 \u03ba\u03b1\u03c4\u03b1\u03b3\u03c1\u03ac\u03c6\u03bf\u03c5\u03bd \u03c9\u03c2 \u03b5\u03c0\u03cc\u03bc\u03b5\u03bd\u03bf \u03b2\u03ae\u03bc\u03b1.<\/p>\n<p>\u0397 \u03c0\u03c1\u03b1\u03ba\u03c4\u03b9\u03ba\u03ae \u03b4\u03b9\u03ac\u03ba\u03c1\u03b9\u03c3\u03b7 \u03b5\u03af\u03bd\u03b1\u03b9 \u03b1\u03bd\u03ac\u03bc\u03b5\u03c3\u03b1 \u03c3\u03b5 scaffolding \u03ba\u03b1\u03b9 \u03bc\u03cc\u03bd\u03b9\u03bc\u03b7 \u03b5\u03be\u03ac\u03c1\u03c4\u03b7\u03c3\u03b7. \u0388\u03bd\u03b1 hint \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b2\u03bf\u03b7\u03b8\u03ae\u03c3\u03b5\u03b9 \u03c3\u03c4\u03b7\u03bd \u03b5\u03be\u03b5\u03c1\u03b5\u03cd\u03bd\u03b7\u03c3\u03b7, \u03b1\u03bb\u03bb\u03ac \u03bf production agent \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03b1\u03be\u03b9\u03bf\u03bb\u03bf\u03b3\u03b5\u03af\u03c4\u03b1\u03b9 \u03c7\u03c9\u03c1\u03af\u03c2 \u03c0\u03c1\u03cc\u03c3\u03b2\u03b1\u03c3\u03b7 \u03c3\u03b5 privileged reference solution. \u0391\u03bd \u03c4\u03bf scaffold \u03c0\u03b1\u03c1\u03b1\u03bc\u03ad\u03bd\u03b5\u03b9 \u03ba\u03c1\u03c5\u03c6\u03cc \u03c3\u03c4\u03b7 \u03b4\u03bf\u03ba\u03b9\u03bc\u03ae, \u03c4\u03bf evaluation \u03bc\u03b5\u03c4\u03c1\u03ac \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03b5\u03c4\u03b9\u03ba\u03cc \u03c3\u03cd\u03c3\u03c4\u03b7\u03bc\u03b1 \u03b1\u03c0\u03cc \u03b5\u03ba\u03b5\u03af\u03bd\u03bf \u03c0\u03bf\u03c5 \u03b8\u03b1 \u03bb\u03b5\u03b9\u03c4\u03bf\u03c5\u03c1\u03b3\u03ae\u03c3\u03b5\u03b9 \u03c3\u03c4\u03b7\u03bd \u03c0\u03b1\u03c1\u03b1\u03b3\u03c9\u03b3\u03ae.<\/p>\n<h2 id=\"critic-loss\">\u039c\u03b9\u03b1 \u03b1\u03bb\u03bb\u03b1\u03b3\u03ae \u03c3\u03c4\u03bf critic loss \u03ac\u03be\u03b9\u03b6\u03b5 14,4 \u03bc\u03bf\u03bd\u03ac\u03b4\u03b5\u03c2<\/h2>\n<p>\u0397 \u03c3\u03cd\u03b3\u03ba\u03c1\u03b9\u03c3\u03b7 VAPO \u03ba\u03b1\u03b9 VAPO-HL \u03b1\u03c0\u03bf\u03bc\u03bf\u03bd\u03ce\u03bd\u03b5\u03b9 \u03c4\u03bf \u03c0\u03b9\u03bf \u03ba\u03b1\u03b8\u03b1\u03c1\u03cc ablation \u03c4\u03b7\u03c2 \u03bc\u03b5\u03bb\u03ad\u03c4\u03b7\u03c2. \u03a4\u03bf \u03c0\u03c1\u03ce\u03c4\u03bf \u03b5\u03af\u03c7\u03b5 39,2% \u03c3\u03c4\u03bf sparse-800, \u03b5\u03bd\u03ce \u03c4\u03bf \u03b4\u03b5\u03cd\u03c4\u03b5\u03c1\u03bf 53,6%: \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03ac 14,4 \u03c0\u03bf\u03c3\u03bf\u03c3\u03c4\u03b9\u03b1\u03af\u03c9\u03bd \u03bc\u03bf\u03bd\u03ac\u03b4\u03c9\u03bd. \u03a3\u03c4\u03bf DynaMath, \u03c4\u03bf VAPO \u03ae\u03c4\u03b1\u03bd \u03c4\u03b5\u03bb\u03b5\u03c5\u03c4\u03b1\u03af\u03bf \u03bc\u03b5\u03c4\u03b1\u03be\u03cd \u03c4\u03c9\u03bd trained arms \u03bc\u03b5 27,0%, \u03b5\u03bd\u03ce \u03c4\u03bf VAPO-HL \u03ad\u03c6\u03c4\u03b1\u03c3\u03b5 31,9%.<\/p>\n<p>\u03a4\u03bf HL-Gauss \u03b4\u03b5\u03bd \u03ac\u03bb\u03bb\u03b1\u03be\u03b5 \u03c4\u03b7\u03bd actor update. \u0391\u03bd\u03c4\u03b9\u03ba\u03b1\u03c4\u03ad\u03c3\u03c4\u03b7\u03c3\u03b5 \u03c4\u03b7 scalar MSE regression \u03c4\u03bf\u03c5 critic \u03bc\u03b5 \u03ba\u03b1\u03c4\u03b7\u03b3\u03bf\u03c1\u03b9\u03ba\u03ae \u03c0\u03c1\u03cc\u03b2\u03bb\u03b5\u03c8\u03b7 \u03c0\u03ac\u03bd\u03c9 \u03c3\u03b5 \u03b4\u03b9\u03b1\u03ba\u03c1\u03b9\u03c4\u03bf\u03c0\u03bf\u03b9\u03b7\u03bc\u03ad\u03bd\u03bf value support \u03ba\u03b1\u03b9 cross-entropy \u03c3\u03b5 smoothed targets. \u03a3\u03c4\u03b7 \u03c3\u03c5\u03b3\u03ba\u03b5\u03ba\u03c1\u03b9\u03bc\u03ad\u03bd\u03b7 \u03c1\u03cd\u03b8\u03bc\u03b9\u03c3\u03b7, \u03b7 \u03b1\u03bb\u03bb\u03b1\u03b3\u03ae \u03b1\u03c6\u03b1\u03af\u03c1\u03b5\u03c3\u03b5 cold-start pathology, \u03b2\u03b5\u03bb\u03c4\u03af\u03c9\u03c3\u03b5 \u03c4\u03bf value learning \u03ba\u03b1\u03b9 \u03b5\u03c0\u03ad\u03c4\u03c1\u03b5\u03c8\u03b5 \u03c3\u03c4\u03bf prior \u03bd\u03b1 \u03c6\u03c4\u03ac\u03c3\u03b5\u03b9 \u03c3\u03c4\u03b7\u03bd policy.<\/p>\n<p>\u03a4\u03bf \u03b1\u03c0\u03bf\u03c4\u03ad\u03bb\u03b5\u03c3\u03bc\u03b1 \u03c3\u03c5\u03bc\u03c6\u03c9\u03bd\u03b5\u03af \u03ba\u03b1\u03c4\u03b5\u03c5\u03b8\u03c5\u03bd\u03c4\u03b9\u03ba\u03ac \u03bc\u03b5 \u03be\u03b5\u03c7\u03c9\u03c1\u03b9\u03c3\u03c4\u03ae \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1 \u03b3\u03b9\u03b1 categorical critics \u03c3\u03b5 LLM reinforcement learning, \u03b1\u03bb\u03bb\u03ac \u03b4\u03b5\u03bd \u03bc\u03b5\u03c4\u03b1\u03c4\u03c1\u03ad\u03c0\u03b5\u03b9 \u03c4\u03bf HL-Gauss \u03c3\u03b5 \u03ba\u03b1\u03b8\u03bf\u03bb\u03b9\u03ba\u03ae \u03bb\u03cd\u03c3\u03b7. \u0394\u03b5\u03af\u03c7\u03bd\u03b5\u03b9 \u03b3\u03b9\u03b1\u03c4\u03af \u03b4\u03b5\u03bd \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03b1\u03c0\u03bf\u03c1\u03c1\u03af\u03c0\u03c4\u03b5\u03c4\u03b1\u03b9 \u03bc\u03b9\u03b1 \u03bf\u03bb\u03cc\u03ba\u03bb\u03b7\u03c1\u03b7 \u03b1\u03c1\u03c7\u03b9\u03c4\u03b5\u03ba\u03c4\u03bf\u03bd\u03b9\u03ba\u03ae \u03b5\u03c0\u03b5\u03b9\u03b4\u03ae \u03b1\u03c0\u03ad\u03c4\u03c5\u03c7\u03b5 \u03bc\u03af\u03b1 loss formulation. \u03a7\u03c1\u03b5\u03b9\u03ac\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 ablations \u03c0\u03bf\u03c5 \u03b1\u03bb\u03bb\u03ac\u03b6\u03bf\u03c5\u03bd \u03ad\u03bd\u03b1 \u03c3\u03c4\u03bf\u03b9\u03c7\u03b5\u03af\u03bf \u03c4\u03b7 \u03c6\u03bf\u03c1\u03ac.<\/p>\n<aside class=\"td-article-note\"><strong>\u039c\u03b7 \u03b4\u03b9\u03b1\u03b2\u03ac\u03b6\u03b5\u03c4\u03b5 \u03c4\u03b9\u03c2 14,4 \u03bc\u03bf\u03bd\u03ac\u03b4\u03b5\u03c2 \u03c9\u03c2 \u03b3\u03b5\u03bd\u03b9\u03ba\u03cc uplift:<\/strong> \u03c4\u03bf \u03bc\u03ad\u03b3\u03b5\u03b8\u03bf\u03c2 \u03c0\u03c1\u03bf\u03ad\u03c1\u03c7\u03b5\u03c4\u03b1\u03b9 \u03b1\u03c0\u03cc \u03ad\u03bd\u03b1 visual-math setup, \u03ad\u03bd\u03b1\u03bd seed \u03ba\u03b1\u03b9 \u03c3\u03c5\u03b3\u03ba\u03b5\u03ba\u03c1\u03b9\u03bc\u03ad\u03bd\u03bf critic. \u0397 \u03b5\u03c0\u03b1\u03bd\u03b1\u03bb\u03b7\u03c0\u03c4\u03b9\u03ba\u03ae \u03b1\u03be\u03af\u03b1 \u03c4\u03bf\u03c5 \u03b5\u03c5\u03c1\u03ae\u03bc\u03b1\u03c4\u03bf\u03c2 \u03b5\u03af\u03bd\u03b1\u03b9 \u03b7 \u03b1\u03bd\u03ac\u03b3\u03ba\u03b7 \u03bd\u03b1 \u03b5\u03bb\u03ad\u03b3\u03c7\u03b5\u03c4\u03b1\u03b9 \u03b7 \u03c0\u03b1\u03c1\u03b1\u03bc\u03b5\u03c4\u03c1\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b7 \u03c4\u03bf\u03c5 value learning, \u03cc\u03c7\u03b9 \u03bd\u03b1 \u03b1\u03bd\u03c4\u03b9\u03b3\u03c1\u03ac\u03c6\u03b5\u03c4\u03b1\u03b9 \u03c4\u03bf \u03c0\u03bf\u03c3\u03bf\u03c3\u03c4\u03cc \u03c3\u03b5 business case.<\/aside>\n<h2 id=\"distillation-ceiling\">\u0397 distillation \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03c0\u03c1\u03cc\u03b3\u03c1\u03b1\u03bc\u03bc\u03b1 \u03ba\u03b1\u03b9 \u03ad\u03c7\u03b5\u03b9 ceiling<\/h2>\n<p>\u03a4\u03bf annealed SAF \u03ae\u03c4\u03b1\u03bd \u03b7 \u03c0\u03b9\u03bf \u03b1\u03bd\u03b8\u03b5\u03ba\u03c4\u03b9\u03ba\u03ae distillation \u03c3\u03c5\u03bd\u03c4\u03b1\u03b3\u03ae \u03c4\u03b7\u03c2 \u03c3\u03cd\u03b3\u03ba\u03c1\u03b9\u03c3\u03b7\u03c2, \u03bc\u03b5 30,6% \u03c3\u03c4\u03bf DynaMath. \u03a7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03bf\u03cd\u03c3\u03b5 bounded teacher-KL signal \u03c0\u03bf\u03c5 \u03b5\u03c6\u03b1\u03c1\u03bc\u03bf\u03b6\u03cc\u03c4\u03b1\u03bd globally \u03ba\u03b1\u03b9 \u03bc\u03b5\u03b9\u03c9\u03bd\u03cc\u03c4\u03b1\u03bd \u03c3\u03c4\u03b1\u03b4\u03b9\u03b1\u03ba\u03ac. \u0391\u03bd\u03c4\u03af\u03b8\u03b5\u03c4\u03b1, \u03c4\u03bf SRPO \u03b5\u03bd\u03b5\u03c1\u03b3\u03bf\u03c0\u03bf\u03b9\u03bf\u03cd\u03c3\u03b5 persistent distillation \u03bc\u03cc\u03bd\u03bf \u03c3\u03c4\u03b1 failed rollouts \u03ba\u03b1\u03b9 \u03ad\u03c6\u03c4\u03b1\u03c3\u03b5 29,1%, \u03c7\u03c9\u03c1\u03af\u03c2 \u03ba\u03b1\u03b8\u03b1\u03c1\u03cc \u03c0\u03bb\u03b5\u03bf\u03bd\u03ad\u03ba\u03c4\u03b7\u03bc\u03b1 \u03ad\u03bd\u03b1\u03bd\u03c4\u03b9 \u03c4\u03bf\u03c5 baseline.<\/p>\n<p>\u0397 \u03bb\u03bf\u03b3\u03b9\u03ba\u03ae \u00ab\u03b4\u03ce\u03c3\u03b5 teacher signal \u03bc\u03cc\u03bd\u03bf \u03c3\u03c4\u03b9\u03c2 \u03b1\u03c0\u03bf\u03c4\u03c5\u03c7\u03af\u03b5\u03c2\u00bb \u03c6\u03b1\u03af\u03bd\u03b5\u03c4\u03b1\u03b9 \u03bf\u03b9\u03ba\u03bf\u03bd\u03bf\u03bc\u03b9\u03ba\u03ae, \u03b1\u03bb\u03bb\u03ac \u03c4\u03bf gate \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b1\u03c6\u03b1\u03b9\u03c1\u03b5\u03af \u03c4\u03bf prior \u03b1\u03ba\u03c1\u03b9\u03b2\u03ce\u03c2 \u03cc\u03c4\u03b1\u03bd \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03b5\u03c5\u03c1\u03cd\u03c4\u03b5\u03c1\u03b7 \u03ba\u03b1\u03c4\u03b5\u03cd\u03b8\u03c5\u03bd\u03c3\u03b7. \u0397 \u03bc\u03b5\u03bb\u03ad\u03c4\u03b7 \u03b4\u03b5\u03af\u03c7\u03bd\u03b5\u03b9 \u03b5\u03c0\u03af\u03c3\u03b7\u03c2 \u03c4\u03bf ceiling \u03c4\u03b7\u03c2 pure distillation: \u03c4\u03bf OPD \u03ba\u03b1\u03b9 \u03c4\u03bf ReOPD \u03bc\u03b9\u03bc\u03bf\u03cd\u03bd\u03c4\u03b1\u03bd teacher \u03c0\u03bf\u03c5 \u03ad\u03c6\u03c4\u03b1\u03bd\u03b5 27,9% \u03c3\u03c4\u03bf pool. \u0397 reverse-KL imitation \u03b4\u03b5\u03bd \u03bc\u03c0\u03bf\u03c1\u03bf\u03cd\u03c3\u03b5 \u03bd\u03b1 \u03be\u03b5\u03c0\u03b5\u03c1\u03ac\u03c3\u03b5\u03b9 \u03b5\u03cd\u03ba\u03bf\u03bb\u03b1 \u03c4\u03b7 \u03c3\u03c5\u03bc\u03c0\u03b5\u03c1\u03b9\u03c6\u03bf\u03c1\u03ac \u03c0\u03bf\u03c5 \u03b1\u03bd\u03c4\u03ad\u03b3\u03c1\u03b1\u03c6\u03b5, \u03b5\u03bd\u03ce \u03c4\u03b1 reward-driven hint arms \u03ad\u03c6\u03c4\u03b1\u03c3\u03b1\u03bd \u03c0\u03b5\u03c1\u03af\u03c0\u03bf\u03c5 52% \u03c3\u03c4\u03bf sparse-800.<\/p>\n<p>\u03a4\u03bf hintsaf \u03b4\u03b5\u03bd \u03c0\u03c1\u03cc\u03c3\u03b8\u03b5\u03c3\u03b5 \u03c4\u03b7 \u03b8\u03b5\u03c9\u03c1\u03b7\u03c4\u03b9\u03ba\u03ac \u03b1\u03bd\u03b1\u03bc\u03b5\u03bd\u03cc\u03bc\u03b5\u03bd\u03b7 \u03c5\u03c0\u03b5\u03c1\u03b1\u03be\u03af\u03b1. \u039c\u03b5 49,4% \u03c3\u03c4\u03bf sparse-800 \u03ba\u03b1\u03b9 30,8% \u03c3\u03c4\u03bf DynaMath \u03b2\u03c1\u03ad\u03b8\u03b7\u03ba\u03b5 \u03c0\u03b5\u03c1\u03af\u03c0\u03bf\u03c5 \u03b1\u03bd\u03ac\u03bc\u03b5\u03c3\u03b1 \u03c3\u03c4\u03bf\u03c5\u03c2 \u03b3\u03bf\u03bd\u03b5\u03af\u03c2 \u03c4\u03bf\u03c5. \u0397 \u03b5\u03c1\u03bc\u03b7\u03bd\u03b5\u03af\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 \u03cc\u03c4\u03b9 hint \u03ba\u03b1\u03b9 annealed teacher signal \u03b1\u03bd\u03c4\u03b9\u03bc\u03b5\u03c4\u03ce\u03c0\u03b9\u03b6\u03b1\u03bd \u03c4\u03bf \u03af\u03b4\u03b9\u03bf \u03c0\u03c1\u03ce\u03b9\u03bc\u03bf bottleneck: \u03c4\u03b7\u03bd \u03ad\u03be\u03bf\u03b4\u03bf \u03b1\u03c0\u03cc \u03c4\u03b7\u03bd \u03c0\u03b5\u03c1\u03b9\u03bf\u03c7\u03ae \u03cc\u03c0\u03bf\u03c5 \u03cc\u03bb\u03b1 \u03c4\u03b1 rollouts \u03b5\u03af\u03bd\u03b1\u03b9 \u03bb\u03ac\u03b8\u03bf\u03c2.<\/p>\n<p>\u0397 \u03c3\u03cd\u03bd\u03b4\u03b5\u03c3\u03b7 \u03bc\u03b5 \u03c4\u03bf <a href=\"https:\/\/twodots.gr\/self-distillation-llms-pi-bias-reasoning\/\">self-distillation \u03ba\u03b1\u03b9 \u03c4\u03bf reasoning<\/a> \u03b5\u03af\u03bd\u03b1\u03b9 \u03bf\u03c5\u03c3\u03b9\u03b1\u03c3\u03c4\u03b9\u03ba\u03ae: \u03b7 \u03bc\u03af\u03bc\u03b7\u03c3\u03b7 \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03bc\u03b5\u03c4\u03b1\u03c6\u03ad\u03c1\u03b5\u03b9 \u03c7\u03c1\u03ae\u03c3\u03b9\u03bc\u03bf prior, \u03b1\u03bb\u03bb\u03ac \u03bc\u03b5\u03c4\u03b1\u03c6\u03ad\u03c1\u03b5\u03b9 \u03b5\u03c0\u03af\u03c3\u03b7\u03c2 \u03c4\u03bf \u03cc\u03c1\u03b9\u03bf \u03ba\u03b1\u03b9 \u03c4\u03b9\u03c2 \u03c0\u03c1\u03bf\u03ba\u03b1\u03c4\u03b1\u03bb\u03ae\u03c8\u03b5\u03b9\u03c2 \u03c4\u03bf\u03c5 teacher. \u03a4\u03bf schedule, \u03b7 \u03b1\u03bd\u03b5\u03be\u03b1\u03c1\u03c4\u03b7\u03c3\u03af\u03b1 \u03c4\u03bf\u03c5 reward \u03ba\u03b1\u03b9 \u03b7 \u03b1\u03be\u03b9\u03bf\u03bb\u03cc\u03b3\u03b7\u03c3\u03b7 \u03c7\u03c9\u03c1\u03af\u03c2 teacher \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03bf\u03cd\u03bd \u03be\u03b5\u03c7\u03c9\u03c1\u03b9\u03c3\u03c4\u03ac design decisions.<\/p>\n<h2 id=\"benchmark-inversion\">\u03a4\u03bf benchmark \u03c0\u03bf\u03c5 \u03b1\u03bd\u03c4\u03ad\u03c3\u03c4\u03c1\u03b5\u03c6\u03b5 \u03c4\u03b7\u03bd \u03ba\u03b1\u03c4\u03ac\u03c4\u03b1\u03be\u03b7<\/h2>\n<p>\u0397 \u03c3\u03b7\u03bc\u03b1\u03bd\u03c4\u03b9\u03ba\u03cc\u03c4\u03b5\u03c1\u03b7 \u03b1\u03bd\u03b1\u03ba\u03ac\u03bb\u03c5\u03c8\u03b7 \u03b1\u03c6\u03bf\u03c1\u03bf\u03cd\u03c3\u03b5 \u03c4\u03bf old-319, \u03ad\u03bd\u03b1 held-out MathV360K slice \u03c0\u03bf\u03c5 \u03bb\u03b5\u03b9\u03c4\u03bf\u03c5\u03c1\u03b3\u03bf\u03cd\u03c3\u03b5 \u03c9\u03c2 \u00ab\u03b3\u03b5\u03bd\u03b9\u03ba\u03cc\u03c2\u00bb \u03ad\u03bb\u03b5\u03b3\u03c7\u03bf\u03c2 \u03c3\u03b5 \u03c0\u03c1\u03bf\u03b7\u03b3\u03bf\u03cd\u03bc\u03b5\u03bd\u03bf\u03c5\u03c2 \u03b3\u03cd\u03c1\u03bf\u03c5\u03c2. \u0397 \u03ba\u03b1\u03c4\u03ac\u03c4\u03b1\u03be\u03ae \u03c4\u03bf\u03c5 \u03b5\u03af\u03c7\u03b5 Spearman \u03c3\u03c5\u03c3\u03c7\u03ad\u03c4\u03b9\u03c3\u03b7 \u22120,74 \u03bc\u03b5 \u03c4\u03bf cross-domain DynaMath \u03b3\u03b9\u03b1 \u03c4\u03b1 \u03ad\u03bd\u03c4\u03b5\u03ba\u03b1 arms, \u03bc\u03b5 permutation p=0,011. \u0391\u03bd\u03c4\u03af\u03b8\u03b5\u03c4\u03b1, \u03c4\u03bf \u03b4\u03cd\u03c3\u03ba\u03bf\u03bb\u03bf sparse-800 \u03b5\u03af\u03c7\u03b5 \u03c3\u03c5\u03c3\u03c7\u03ad\u03c4\u03b9\u03c3\u03b7 +0,89, \u03bc\u03b5 p \u03bc\u03b9\u03ba\u03c1\u03cc\u03c4\u03b5\u03c1\u03bf \u03b1\u03c0\u03cc 0,001.<\/p>\n<p>\u03a4\u03bf 69% \u03c4\u03bf\u03c5 old-319 \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03bf\u03cd\u03bd\u03c4\u03b1\u03bd \u03b1\u03c0\u03cc Geometry3K \u03b5\u03c1\u03c9\u03c4\u03ae\u03c3\u03b5\u03b9\u03c2 \u03c4\u03b5\u03c3\u03c3\u03ac\u03c1\u03c9\u03bd \u03b5\u03c0\u03b9\u03bb\u03bf\u03b3\u03ce\u03bd. \u038c\u03bb\u03b1 \u03c4\u03b1 arms \u03ba\u03b9\u03bd\u03bf\u03cd\u03bd\u03c4\u03b1\u03bd \u03ba\u03bf\u03bd\u03c4\u03ac \u03c3\u03c4\u03bf \u03c4\u03c5\u03c7\u03b1\u03af\u03bf \u03b5\u03c0\u03af\u03c0\u03b5\u03b4\u03bf 25%. \u0397 \u03b5\u03c1\u03bc\u03b7\u03bd\u03b5\u03af\u03b1 \u03c4\u03c9\u03bd \u03c3\u03c5\u03b3\u03b3\u03c1\u03b1\u03c6\u03ad\u03c9\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03cc\u03c4\u03b9 \u03ad\u03bd\u03b1 \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03bf \u03c0\u03bf\u03c5 \u03ad\u03bc\u03b5\u03bd\u03b5 \u03ba\u03bf\u03bd\u03c4\u03ac \u03c3\u03c4\u03bf base behavior \u03c3\u03c5\u03bd\u03ad\u03c7\u03b9\u03b6\u03b5 \u03bd\u03b1 \u03b5\u03ba\u03c0\u03ad\u03bc\u03c0\u03b5\u03b9 \u03c3\u03cd\u03bd\u03c4\u03bf\u03bc\u03b1 \u03b3\u03c1\u03ac\u03bc\u03bc\u03b1\u03c4\u03b1 \u03b5\u03c0\u03b9\u03bb\u03bf\u03b3\u03ce\u03bd \u03ba\u03b1\u03b9 \u03bc\u03c0\u03bf\u03c1\u03bf\u03cd\u03c3\u03b5 \u03bd\u03b1 \u03b4\u03b9\u03b1\u03c4\u03b7\u03c1\u03b5\u03af \u03ba\u03b1\u03bb\u03cd\u03c4\u03b5\u03c1\u03bf score, \u03b5\u03bd\u03ce \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03b1 \u03c0\u03bf\u03c5 \u03b5\u03af\u03c7\u03b1\u03bd \u03b1\u03bb\u03bb\u03ac\u03be\u03b5\u03b9 \u03c0\u03c1\u03bf\u03c2 \u03c0\u03bf\u03bb\u03c5\u03b2\u03ae\u03bc\u03b1\u03c4\u03b7 \u03c3\u03c5\u03bb\u03bb\u03bf\u03b3\u03b9\u03c3\u03c4\u03b9\u03ba\u03ae \u03ad\u03c7\u03b1\u03bd\u03b1\u03bd \u03c4\u03c5\u03c7\u03b1\u03af\u03b5\u03c2 \u03b5\u03c0\u03b9\u03c4\u03c5\u03c7\u03af\u03b5\u03c2.<\/p>\n<p>\u03a4\u03bf VAPO \u03b5\u03af\u03c7\u03b5 \u03c4\u03bf \u03c5\u03c8\u03b7\u03bb\u03cc\u03c4\u03b5\u03c1\u03bf old-319 score, 32,6%, \u03b1\u03bb\u03bb\u03ac \u03c4\u03bf \u03c7\u03b1\u03bc\u03b7\u03bb\u03cc\u03c4\u03b5\u03c1\u03bf DynaMath score \u03bc\u03b5\u03c4\u03b1\u03be\u03cd \u03c4\u03c9\u03bd trained arms, 27,0%. \u03a4\u03bf VAPO-HL \u03b5\u03af\u03c7\u03b5 \u03bc\u03cc\u03bb\u03b9\u03c2 19,7% \u03c3\u03c4\u03bf old-319, \u03b1\u03bb\u03bb\u03ac 31,9% \u03c3\u03c4\u03bf DynaMath. \u0391\u03bd \u03b7 \u03b1\u03c0\u03cc\u03c6\u03b1\u03c3\u03b7 \u03b2\u03b1\u03c3\u03b9\u03b6\u03cc\u03c4\u03b1\u03bd \u03bc\u03cc\u03bd\u03bf \u03c3\u03c4\u03bf \u03c0\u03b1\u03bb\u03b9\u03cc held-out set, \u03b7 \u03ba\u03b1\u03c4\u03ac\u03c4\u03b1\u03be\u03b7 \u03c4\u03c9\u03bd \u03b4\u03cd\u03bf methods \u03b8\u03b1 \u03ae\u03c4\u03b1\u03bd \u03c3\u03c7\u03b5\u03b4\u03cc\u03bd \u03b1\u03bd\u03c4\u03af\u03c3\u03c4\u03c1\u03bf\u03c6\u03b7.<\/p>\n<div class=\"td-decision-band\">\n<p class=\"td-decision-kicker\">\u039a\u03b1\u03bd\u03cc\u03bd\u03b1\u03c2 \u03b1\u03c0\u03cc\u03c6\u03b1\u03c3\u03b7\u03c2 \u03b3\u03b9\u03b1 AI benchmarks<\/p>\n<p class=\"td-decision-title\">\u039c\u03b7\u03bd \u03b5\u03c0\u03b9\u03bb\u03ad\u03b3\u03b5\u03c4\u03b5 \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03bf \u03c0\u03c1\u03b9\u03bd \u03b5\u03bb\u03ad\u03b3\u03be\u03b5\u03c4\u03b5 \u03c0\u03bf\u03b9\u03b1 slices \u03c0\u03b1\u03c1\u03ac\u03b3\u03bf\u03c5\u03bd \u03c4\u03b7\u03bd \u03ba\u03b1\u03c4\u03ac\u03c4\u03b1\u03be\u03b7<\/p>\n<p>Go \u03c3\u03b7\u03bc\u03b1\u03af\u03bd\u03b5\u03b9 semantic scorer, \u03b1\u03bd\u03ac\u03bb\u03c5\u03c3\u03b7 \u03b1\u03bd\u03ac slice, \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc cross-domain set, blind judging, paired tests \u03ba\u03b1\u03b9 \u03b5\u03c0\u03b1\u03bd\u03ac\u03bb\u03b7\u03c8\u03b7 \u03c3\u03b5 \u03bd\u03ad\u03b1 \u03b4\u03b5\u03b4\u03bf\u03bc\u03ad\u03bd\u03b1. No-go \u03c3\u03b7\u03bc\u03b1\u03af\u03bd\u03b5\u03b9 \u03ad\u03bd\u03b1 pooled accuracy, extractor \u03b5\u03c5\u03b1\u03af\u03c3\u03b8\u03b7\u03c4\u03bf\u03c2 \u03c3\u03c4\u03bf format, near-chance multiple-choice \u03c0\u03bb\u03b5\u03b9\u03bf\u03c8\u03b7\u03c6\u03af\u03b1 \u03ae held-out set \u03c0\u03bf\u03c5 \u03b4\u03b5\u03bd \u03ad\u03c7\u03b5\u03b9 \u03c3\u03c5\u03b3\u03ba\u03c1\u03b9\u03b8\u03b5\u03af \u03bc\u03b5 \u03c4\u03bf \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc \u03c0\u03b5\u03c1\u03b9\u03b2\u03ac\u03bb\u03bb\u03bf\u03bd \u03c7\u03c1\u03ae\u03c3\u03b7\u03c2.<\/p>\n<\/div>\n<p>\u0397 \u03bc\u03b5\u03bb\u03ad\u03c4\u03b7 \u03b3\u03b9\u03b1 \u03c4\u03bf <a href=\"https:\/\/twodots.gr\/ai-benchmark-harness-allazei-nikiti\/\">\u03c0\u03ce\u03c2 \u03c4\u03bf evaluation harness \u03b1\u03bb\u03bb\u03ac\u03b6\u03b5\u03b9 \u03c4\u03bf\u03bd \u03bd\u03b9\u03ba\u03b7\u03c4\u03ae<\/a> \u03c6\u03c9\u03c4\u03af\u03b6\u03b5\u03b9 \u03c4\u03b7\u03bd \u03af\u03b4\u03b9\u03b1 \u03b1\u03c1\u03c7\u03ae \u03b1\u03c0\u03cc \u03c0\u03bb\u03b5\u03c5\u03c1\u03ac\u03c2 \u03c5\u03c0\u03bf\u03b4\u03bf\u03bc\u03ae\u03c2. \u0395\u03b4\u03ce \u03b7 \u03b1\u03bd\u03b1\u03c4\u03c1\u03bf\u03c0\u03ae \u03c0\u03c1\u03bf\u03ae\u03bb\u03b8\u03b5 \u03b1\u03c0\u03cc \u03c4\u03b7 \u03c3\u03cd\u03bd\u03b8\u03b5\u03c3\u03b7 \u03c4\u03bf\u03c5 benchmark \u03ba\u03b1\u03b9 \u03c4\u03b7 \u03c3\u03c5\u03bc\u03c0\u03b5\u03c1\u03b9\u03c6\u03bf\u03c1\u03ac \u03c4\u03bf\u03c5 scorer. \u039a\u03b1\u03b9 \u03c3\u03c4\u03b9\u03c2 \u03b4\u03cd\u03bf \u03c0\u03b5\u03c1\u03b9\u03c0\u03c4\u03ce\u03c3\u03b5\u03b9\u03c2, \u03c4\u03bf leaderboard \u03b5\u03af\u03bd\u03b1\u03b9 \u03c0\u03c1\u03bf\u03ca\u03cc\u03bd \u03bf\u03bb\u03cc\u03ba\u03bb\u03b7\u03c1\u03bf\u03c5 \u03c4\u03bf\u03c5 measurement system.<\/p>\n<h2 id=\"verification-audit\">Scoring audit, blind judging \u03ba\u03b1\u03b9 hard gates<\/h2>\n<p>\u039f\u03b9 \u03c3\u03c5\u03b3\u03b3\u03c1\u03b1\u03c6\u03b5\u03af\u03c2 \u03b1\u03bd\u03b1\u03c6\u03ad\u03c1\u03bf\u03c5\u03bd \u03cc\u03c4\u03b9 \u03b4\u03cd\u03bf \u03c0\u03c1\u03bf\u03b7\u03b3\u03bf\u03cd\u03bc\u03b5\u03bd\u03bf\u03b9 \u03b3\u03cd\u03c1\u03bf\u03b9 \u03b5\u03af\u03c7\u03b1\u03bd \u03b1\u03bd\u03b1\u03ba\u03bb\u03b7\u03b8\u03b5\u03af \u03bc\u03b5\u03c4\u03ac \u03b1\u03c0\u03cc \u03c0\u03ad\u03bd\u03c4\u03b5 silent port-fidelity bugs, \u03bc\u03b5 \u03b4\u03b9\u03bf\u03c1\u03b8\u03ce\u03c3\u03b5\u03b9\u03c2 \u03c0\u03bf\u03c5 \u03bc\u03b5\u03c4\u03b1\u03ba\u03af\u03bd\u03b7\u03c3\u03b1\u03bd \u03bc\u03b5\u03bc\u03bf\u03bd\u03c9\u03bc\u03ad\u03bd\u03b1 scores \u03ad\u03c9\u03c2 9,1 \u03bc\u03bf\u03bd\u03ac\u03b4\u03b5\u03c2. \u03a3\u03c4\u03bf\u03bd \u03c4\u03ad\u03c4\u03b1\u03c1\u03c4\u03bf \u03b3\u03cd\u03c1\u03bf \u03b1\u03bd\u03c4\u03b9\u03bc\u03b5\u03c4\u03ce\u03c0\u03b9\u03c3\u03b1\u03bd \u03c4\u03b7\u03bd \u03b5\u03c0\u03b1\u03bb\u03ae\u03b8\u03b5\u03c5\u03c3\u03b7 \u03c9\u03c2 \u03c0\u03b1\u03c1\u03b1\u03b4\u03bf\u03c4\u03ad\u03bf: scoring audit, pre-training gates, blind judging \u03ba\u03b1\u03b9 reward-hacking analyses \u03bc\u03c0\u03ae\u03ba\u03b1\u03bd \u03c0\u03c1\u03b9\u03bd \u03b1\u03c0\u03cc \u03c4\u03b1 \u03c3\u03c5\u03bc\u03c0\u03b5\u03c1\u03ac\u03c3\u03bc\u03b1\u03c4\u03b1.<\/p>\n<p>\u039f heuristic extractor \u03c4\u03b9\u03bc\u03c9\u03c1\u03bf\u03cd\u03c3\u03b5 \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03b5\u03c4\u03b9\u03ba\u03ac arms \u03ba\u03b1\u03c4\u03ac 19 \u03ad\u03c9\u03c2 30 \u03bc\u03bf\u03bd\u03ac\u03b4\u03b5\u03c2 \u03ba\u03b1\u03b9 \u03bc\u03c0\u03bf\u03c1\u03bf\u03cd\u03c3\u03b5 \u03bd\u03b1 \u03b1\u03bb\u03bb\u03ac\u03be\u03b5\u03b9 \u03c3\u03c7\u03b5\u03b4\u03cc\u03bd \u03ba\u03ac\u03b8\u03b5 pairwise \u03c3\u03cd\u03b3\u03ba\u03c1\u03b9\u03c3\u03b7. \u0393\u03b9\u03b1 \u03c0\u03b1\u03c1\u03ac\u03b4\u03b5\u03b9\u03b3\u03bc\u03b1, \u03bc\u03bf\u03c1\u03c6\u03ad\u03c2 \u03cc\u03c0\u03c9\u03c2 \u00abFinal answer: A. 5.\u00bb \u03b4\u03b5\u03bd \u03b1\u03bd\u03b1\u03b3\u03bd\u03c9\u03c1\u03af\u03b6\u03bf\u03bd\u03c4\u03b1\u03bd \u03c3\u03c9\u03c3\u03c4\u03ac \u03c3\u03b5 multiple-choice rows. \u039c\u03cc\u03bd\u03bf \u03bf\u03b9 blind-judged \u03b1\u03c1\u03b9\u03b8\u03bc\u03bf\u03af \u03bc\u03c0\u03ae\u03ba\u03b1\u03bd \u03c3\u03c4\u03bf\u03bd \u03c4\u03b5\u03bb\u03b9\u03ba\u03cc \u03c0\u03af\u03bd\u03b1\u03ba\u03b1.<\/p>\n<p>\u039f\u03b9 paired claims \u03b5\u03bb\u03ad\u03b3\u03c7\u03b8\u03b7\u03ba\u03b1\u03bd \u03bc\u03b5 exact McNemar tests. \u03a4\u03c1\u03b5\u03b9\u03c2 \u03b1\u03bd\u03b5\u03be\u03ac\u03c1\u03c4\u03b7\u03c4\u03b5\u03c2 judging passes \u03c3\u03c4\u03b1 \u03af\u03b4\u03b9\u03b1 GRPO outputs \u03ad\u03b4\u03c9\u03c3\u03b1\u03bd 40,5%, 41,1% \u03ba\u03b1\u03b9 41,0%, \u03ac\u03c1\u03b1 \u03b7 judge variance \u03ae\u03c4\u03b1\u03bd \u03c0\u03b5\u03c1\u03af\u03c0\u03bf\u03c5 \u00b10,5 \u03bc\u03bf\u03bd\u03ac\u03b4\u03b1. \u0391\u03c5\u03c4\u03cc \u03b2\u03bf\u03b7\u03b8\u03ac \u03bd\u03b1 \u03be\u03b5\u03c7\u03c9\u03c1\u03af\u03c3\u03b5\u03b9 \u03b7 \u03bc\u03b9\u03ba\u03c1\u03ae \u03b4\u03b9\u03b1\u03ba\u03cd\u03bc\u03b1\u03bd\u03c3\u03b7 \u03c4\u03bf\u03c5 scorer \u03b1\u03c0\u03cc \u03c4\u03b9\u03c2 \u03bc\u03b5\u03b3\u03b1\u03bb\u03cd\u03c4\u03b5\u03c1\u03b5\u03c2 \u03bc\u03b5\u03c4\u03b1\u03be\u03cd-group \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03ad\u03c2.<\/p>\n<p>\u03a4\u03b1 \u03c3\u03c4\u03b1\u03c4\u03b9\u03c3\u03c4\u03b9\u03ba\u03ac tests \u03b4\u03b5\u03bd \u03b4\u03b9\u03bf\u03c1\u03b8\u03ce\u03bd\u03bf\u03c5\u03bd \u03bb\u03ac\u03b8\u03bf\u03c2 measurement pipeline. \u039c\u03b9\u03b1 \u03c0\u03bf\u03bb\u03cd \u03bc\u03b9\u03ba\u03c1\u03ae p-value \u03b1\u03c0\u03b1\u03bd\u03c4\u03ac \u03bc\u03b5 \u03b1\u03ba\u03c1\u03af\u03b2\u03b5\u03b9\u03b1 \u03c3\u03c4\u03b7\u03bd \u03b5\u03c1\u03ce\u03c4\u03b7\u03c3\u03b7 \u03c0\u03bf\u03c5 \u03c4\u03b7\u03c2 \u03b4\u03cc\u03b8\u03b7\u03ba\u03b5\u00b7 \u03b4\u03b5\u03bd \u03b1\u03c0\u03bf\u03b4\u03b5\u03b9\u03ba\u03bd\u03cd\u03b5\u03b9 \u03cc\u03c4\u03b9 \u03bf scorer, \u03c4\u03bf slice \u03ae \u03b7 \u03b5\u03c1\u03ce\u03c4\u03b7\u03c3\u03b7 \u03b5\u03af\u03bd\u03b1\u03b9 \u03c4\u03b1 \u03c3\u03c9\u03c3\u03c4\u03ac. \u0393\u03b9\u03b1 \u03b1\u03c5\u03c4\u03cc \u03c4\u03b1 hard gates \u03ba\u03b1\u03b9 \u03b7 \u03b5\u03c0\u03b9\u03b8\u03b5\u03ce\u03c1\u03b7\u03c3\u03b7 \u03c4\u03c9\u03bd \u03b1\u03c0\u03bf\u03c4\u03c5\u03c7\u03b7\u03bc\u03ad\u03bd\u03c9\u03bd rows \u03c0\u03c1\u03bf\u03b7\u03b3\u03bf\u03cd\u03bd\u03c4\u03b1\u03b9 \u03c4\u03bf\u03c5 leaderboard.<\/p>\n<h2 id=\"business-ai\">\u03a4\u03b9 \u03c3\u03b7\u03bc\u03b1\u03af\u03bd\u03b5\u03b9 \u03b3\u03b9\u03b1 \u03c0\u03c1\u03bf\u03ca\u03cc\u03bd\u03c4\u03b1, e-commerce \u03ba\u03b1\u03b9 marketing AI<\/h2>\n<p>\u0397 \u03b5\u03c1\u03b3\u03b1\u03c3\u03af\u03b1 \u03b4\u03b5\u03bd \u03b4\u03bf\u03ba\u03b9\u03bc\u03ac\u03b6\u03b5\u03b9 \u03ba\u03b1\u03c4\u03b1\u03c3\u03c4\u03ae\u03bc\u03b1\u03c4\u03b1, CRM agents \u03ae \u03ba\u03b1\u03bc\u03c0\u03ac\u03bd\u03b9\u03b5\u03c2. \u039f\u03b9 \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03b7\u03c3\u03b9\u03b1\u03ba\u03ad\u03c2 \u03b5\u03c6\u03b1\u03c1\u03bc\u03bf\u03b3\u03ad\u03c2 \u03b5\u03af\u03bd\u03b1\u03b9 \u03b5\u03c0\u03bf\u03bc\u03ad\u03bd\u03c9\u03c2 \u03bc\u03b5\u03c4\u03b1\u03c6\u03bf\u03c1\u03ac \u03c4\u03b7\u03c2 \u03bc\u03b5\u03b8\u03bf\u03b4\u03bf\u03bb\u03bf\u03b3\u03af\u03b1\u03c2, \u03cc\u03c7\u03b9 \u03ac\u03bc\u03b5\u03c3\u03bf empirical claim. \u038c\u03c4\u03b1\u03bd \u03ad\u03bd\u03b1\u03c2 agent \u03b1\u03be\u03b9\u03bf\u03bb\u03bf\u03b3\u03b5\u03af\u03c4\u03b1\u03b9 \u03bc\u03cc\u03bd\u03bf \u03b1\u03c0\u03cc \u03ad\u03bd\u03b1 \u03c3\u03c0\u03ac\u03bd\u03b9\u03bf \u03c4\u03b5\u03bb\u03b9\u03ba\u03cc conversion, \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03bc\u03b7 \u03bb\u03b1\u03bc\u03b2\u03ac\u03bd\u03b5\u03b9 \u03b1\u03c1\u03ba\u03b5\u03c4\u03cc \u03c3\u03ae\u03bc\u03b1 \u03b3\u03b9\u03b1 \u03c4\u03bf \u03c0\u03bf\u03b9\u03b1 \u03b5\u03bd\u03b4\u03b9\u03ac\u03bc\u03b5\u03c3\u03b1 \u03b2\u03ae\u03bc\u03b1\u03c4\u03b1 \u03ae\u03c4\u03b1\u03bd \u03c3\u03c9\u03c3\u03c4\u03ac.<\/p>\n<p>\u03a3\u03b5 e-commerce, \u03ad\u03bd\u03b1 bot \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03bd\u03b1 \u03b5\u03bd\u03c4\u03bf\u03c0\u03af\u03c3\u03b5\u03b9 \u03c0\u03c1\u03bf\u03ca\u03cc\u03bd, \u03bd\u03b1 \u03b5\u03bb\u03ad\u03b3\u03be\u03b5\u03b9 \u03b4\u03b9\u03b1\u03b8\u03b5\u03c3\u03b9\u03bc\u03cc\u03c4\u03b7\u03c4\u03b1, \u03bd\u03b1 \u03b5\u03c6\u03b1\u03c1\u03bc\u03cc\u03c3\u03b5\u03b9 \u03c0\u03bf\u03bb\u03b9\u03c4\u03b9\u03ba\u03ae \u03b5\u03c0\u03b9\u03c3\u03c4\u03c1\u03bf\u03c6\u03ce\u03bd \u03ba\u03b1\u03b9 \u03bd\u03b1 \u03bf\u03bb\u03bf\u03ba\u03bb\u03b7\u03c1\u03ce\u03c3\u03b5\u03b9 handoff \u03c0\u03c1\u03b9\u03bd \u03c5\u03c0\u03ac\u03c1\u03be\u03b5\u03b9 \u03c4\u03bf \u03c4\u03b5\u03bb\u03b9\u03ba\u03cc outcome. \u0391\u03bd \u03c4\u03bf reward \u03ad\u03c1\u03c7\u03b5\u03c4\u03b1\u03b9 \u03bc\u03cc\u03bd\u03bf \u03bc\u03b5\u03c4\u03ac \u03c4\u03b7\u03bd \u03b1\u03b3\u03bf\u03c1\u03ac, \u03b1\u03c0\u03bf\u03c4\u03c5\u03c7\u03b7\u03bc\u03ad\u03bd\u03b1 sessions \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03b5\u03c4\u03b9\u03ba\u03ae\u03c2 \u03c0\u03bf\u03b9\u03cc\u03c4\u03b7\u03c4\u03b1\u03c2 \u03c3\u03c5\u03bc\u03c0\u03b9\u03ad\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 \u03c3\u03c4\u03bf \u03af\u03b4\u03b9\u03bf \u03bc\u03b7\u03b4\u03ad\u03bd. Hints, teacher feedback \u03ae value estimates \u03bc\u03c0\u03bf\u03c1\u03bf\u03cd\u03bd \u03bd\u03b1 \u03bb\u03b5\u03b9\u03c4\u03bf\u03c5\u03c1\u03b3\u03ae\u03c3\u03bf\u03c5\u03bd \u03c9\u03c2 scaffolding, \u03b1\u03bb\u03bb\u03ac \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03bc\u03b5\u03c4\u03c1\u03b9\u03ad\u03c4\u03b1\u03b9 \u03b1\u03bd \u03b2\u03b5\u03bb\u03c4\u03b9\u03ce\u03bd\u03bf\u03c5\u03bd \u03c4\u03b7\u03bd \u03b1\u03c5\u03c4\u03cc\u03bd\u03bf\u03bc\u03b7 policy.<\/p>\n<p>\u0388\u03bd\u03b1 held-out set \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 \u03c4\u03b5\u03c7\u03bd\u03b9\u03ba\u03ac \u03b1\u03bd\u03b5\u03be\u03ac\u03c1\u03c4\u03b7\u03c4\u03bf \u03ba\u03b1\u03b9 \u03c0\u03b1\u03c1\u2019 \u03cc\u03bb\u03b1 \u03b1\u03c5\u03c4\u03ac \u03ba\u03b1\u03ba\u03cc\u03c2 \u03b4\u03b5\u03af\u03ba\u03c4\u03b7\u03c2 \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03ae\u03c2 \u03c7\u03c1\u03ae\u03c3\u03b7\u03c2. \u0388\u03bd\u03b1 support test \u03c5\u03c0\u03b5\u03c1\u03c6\u03bf\u03c1\u03c4\u03c9\u03bc\u03ad\u03bd\u03bf \u03bc\u03b5 \u03b5\u03cd\u03ba\u03bf\u03bb\u03b5\u03c2 template questions \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b5\u03c0\u03b9\u03b2\u03c1\u03b1\u03b2\u03b5\u03cd\u03b5\u03b9 bot \u03c0\u03bf\u03c5 \u03b4\u03b5\u03bd \u03ac\u03bb\u03bb\u03b1\u03be\u03b5, \u03b5\u03bd\u03ce \u03b1\u03c0\u03bf\u03c4\u03c5\u03b3\u03c7\u03ac\u03bd\u03b5\u03b9 \u03c3\u03b5 \u03bd\u03ad\u03b1 \u03c0\u03c1\u03bf\u03ca\u03cc\u03bd\u03c4\u03b1, \u03c3\u03cd\u03bd\u03b8\u03b5\u03c4\u03b5\u03c2 \u03b5\u03c0\u03b9\u03c3\u03c4\u03c1\u03bf\u03c6\u03ad\u03c2 \u03ae \u03b1\u03c3\u03b1\u03c6\u03ae intents. \u0397 \u03c3\u03c9\u03c3\u03c4\u03ae \u03b4\u03b9\u03b1\u03b4\u03b9\u03ba\u03b1\u03c3\u03af\u03b1 \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03b4\u03cd\u03c3\u03ba\u03bf\u03bb\u03bf in-domain tail, \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc cross-domain set \u03ba\u03b1\u03b9 \u03b1\u03bd\u03ac\u03bb\u03c5\u03c3\u03b7 \u03b1\u03bd\u03ac slice.<\/p>\n<p>\u03a4\u03bf \u03c4\u03c1\u03af\u03c4\u03bf \u03bc\u03ac\u03b8\u03b7\u03bc\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 \u03b7 \u03c0\u03b1\u03c1\u03b1\u03c4\u03b7\u03c1\u03b7\u03c3\u03b9\u03bc\u03cc\u03c4\u03b7\u03c4\u03b1. \u039f\u03b9 \u03bf\u03bc\u03ac\u03b4\u03b5\u03c2 \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03bd\u03b1 \u03b3\u03bd\u03c9\u03c1\u03af\u03b6\u03bf\u03c5\u03bd \u03c0\u03cc\u03c3\u03b1 training groups \u03ad\u03b4\u03c9\u03c3\u03b1\u03bd \u03bc\u03b7\u03b4\u03b5\u03bd\u03b9\u03ba\u03ae \u03c0\u03bb\u03b7\u03c1\u03bf\u03c6\u03bf\u03c1\u03af\u03b1, \u03c0\u03cc\u03c4\u03b5 \u03b5\u03bd\u03b5\u03c1\u03b3\u03bf\u03c0\u03bf\u03b9\u03ae\u03b8\u03b7\u03ba\u03b5 \u03c4\u03bf teacher signal, \u03b1\u03bd \u03bf critic \u03c0\u03b1\u03c1\u03ae\u03b3\u03b1\u03b3\u03b5 \u03c7\u03c1\u03ae\u03c3\u03b9\u03bc\u03b7 \u03b4\u03b9\u03b1\u03ba\u03cd\u03bc\u03b1\u03bd\u03c3\u03b7 \u03ba\u03b1\u03b9 \u03c0\u03bf\u03b9\u03b1 \u03c5\u03c0\u03bf\u03c3\u03cd\u03bd\u03bf\u03bb\u03b1 \u03ba\u03b1\u03b8\u03bf\u03c1\u03af\u03b6\u03bf\u03c5\u03bd \u03c4\u03b7\u03bd \u03ba\u03b1\u03c4\u03ac\u03c4\u03b1\u03be\u03b7. \u03a4\u03bf <a href=\"https:\/\/twodots.gr\/ai-agents-behavioral-testing\/\">behavioral testing \u03b3\u03b9\u03b1 AI agents<\/a> \u03c3\u03c5\u03bc\u03c0\u03bb\u03b7\u03c1\u03ce\u03bd\u03b5\u03b9 \u03b1\u03c5\u03c4\u03ae \u03c4\u03b7\u03bd \u03b5\u03b9\u03ba\u03cc\u03bd\u03b1: \u03c4\u03bf aggregate success rate \u03b4\u03b5\u03bd \u03b1\u03c1\u03ba\u03b5\u03af \u03cc\u03c4\u03b1\u03bd \u03bf\u03b9 failure modes \u03b1\u03bb\u03bb\u03ac\u03b6\u03bf\u03c5\u03bd \u03b1\u03bd\u03ac \u03c3\u03c5\u03bc\u03c0\u03b5\u03c1\u03b9\u03c6\u03bf\u03c1\u03ac.<\/p>\n<h2 id=\"implementation-checklist\">\u0388\u03be\u03b9 \u03ad\u03bb\u03b5\u03b3\u03c7\u03bf\u03b9 \u03b3\u03b9\u03b1 \u03b4\u03cd\u03c3\u03ba\u03bf\u03bb\u03bf AI evaluation<\/h2>\n<div class=\"td-step-list\">\n<p class=\"td-step-list-title\">\u0391\u03c0\u03cc \u03c4\u03bf learning signal \u03c3\u03b5 audit-ready \u03b1\u03c0\u03cc\u03c6\u03b1\u03c3\u03b7<\/p>\n<ol>\n<li><span class=\"td-step-kicker\">\u0421\u0442\u044a\u043f\u043a\u0430 1<\/span><strong>\u039c\u03b5\u03c4\u03c1\u03ae\u03c3\u03c4\u03b5 \u03c0\u03cc\u03c3\u03bf sparse \u03b5\u03af\u03bd\u03b1\u03b9 \u03c4\u03bf reward<\/strong>\n<p>\u039a\u03b1\u03c4\u03b1\u03b3\u03c1\u03ac\u03c8\u03c4\u03b5 \u03c4\u03bf \u03c0\u03bf\u03c3\u03bf\u03c3\u03c4\u03cc all-wrong groups, \u03c4\u03b7 \u03b4\u03b9\u03b1\u03ba\u03cd\u03bc\u03b1\u03bd\u03c3\u03b7 \u03c4\u03bf\u03c5 reward \u03ba\u03b1\u03b9 \u03c0\u03cc\u03c3\u03b1 batches \u03c3\u03c5\u03bd\u03b5\u03b9\u03c3\u03c6\u03ad\u03c1\u03bf\u03c5\u03bd \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc task gradient \u03c0\u03c1\u03b9\u03bd \u03b1\u03bb\u03bb\u03ac\u03be\u03b5\u03c4\u03b5 optimizer \u03ae \u03b1\u03c5\u03be\u03ae\u03c3\u03b5\u03c4\u03b5 compute.<\/p>\n<\/li>\n<li><span class=\"td-step-kicker\">\u0421\u0442\u044a\u043f\u043a\u0430 2<\/span><strong>\u0395\u03bb\u03ad\u03b3\u03be\u03c4\u03b5 semantic correctness \u03ba\u03b1\u03b9 format \u03be\u03b5\u03c7\u03c9\u03c1\u03b9\u03c3\u03c4\u03ac<\/strong>\n<p>\u03a3\u03c5\u03b3\u03ba\u03c1\u03af\u03bd\u03b5\u03c4\u03b5 \u03c4\u03bf\u03bd production scorer \u03bc\u03b5 \u03c7\u03b5\u03b9\u03c1\u03bf\u03ba\u03af\u03bd\u03b7\u03c4\u03bf \u03ae blind semantic audit \u03ba\u03b1\u03b9 \u03ba\u03c1\u03b1\u03c4\u03ae\u03c3\u03c4\u03b5 \u03be\u03b5\u03c7\u03c9\u03c1\u03b9\u03c3\u03c4\u03cc metric \u03b3\u03b9\u03b1 \u03c4\u03b7 \u03c3\u03c5\u03bc\u03bc\u03cc\u03c1\u03c6\u03c9\u03c3\u03b7 \u03c3\u03c4\u03bf \u03b1\u03c0\u03b1\u03b9\u03c4\u03bf\u03cd\u03bc\u03b5\u03bd\u03bf output contract.<\/p>\n<\/li>\n<li><span class=\"td-step-kicker\">\u0421\u0442\u044a\u043f\u043a\u0430 3<\/span><strong>\u0391\u03c0\u03bf\u03b4\u03b5\u03af\u03be\u03c4\u03b5 \u03cc\u03c4\u03b9 \u03c4\u03bf prior \u03c0\u03b1\u03c1\u03b1\u03b4\u03af\u03b4\u03b5\u03c4\u03b1\u03b9<\/strong>\n<p>\u039c\u03b5\u03c4\u03c1\u03ae\u03c3\u03c4\u03b5 activation rate, gradient contribution, teacher agreement \u03ba\u03b1\u03b9 \u03b1\u03bb\u03bb\u03b1\u03b3\u03ae \u03c4\u03b7\u03c2 \u03c4\u03b5\u03bb\u03b9\u03ba\u03ae\u03c2 policy\u00b7 \u03ad\u03bd\u03b1 configured loss \u03ae module \u03b4\u03b5\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03b1\u03c0\u03cc \u03bc\u03cc\u03bd\u03bf \u03c4\u03bf\u03c5 evidence.<\/p>\n<\/li>\n<li><span class=\"td-step-kicker\">\u0421\u0442\u044a\u043f\u043a\u0430 4<\/span><strong>\u03a4\u03c1\u03ad\u03be\u03c4\u03b5 ablations \u03b1\u03bd\u03ac \u03bc\u03b7\u03c7\u03b1\u03bd\u03b9\u03c3\u03bc\u03cc<\/strong>\n<p>\u0391\u03c0\u03bf\u03bc\u03bf\u03bd\u03ce\u03c3\u03c4\u03b5 exploration, auxiliary objective, teacher schedule \u03ba\u03b1\u03b9 critic loss, \u03ce\u03c3\u03c4\u03b5 \u03b7 \u03bf\u03bc\u03ac\u03b4\u03b1 \u03bd\u03b1 \u03b3\u03bd\u03c9\u03c1\u03af\u03b6\u03b5\u03b9 \u03c0\u03bf\u03b9\u03bf \u03c3\u03c4\u03bf\u03b9\u03c7\u03b5\u03af\u03bf \u03c0\u03b1\u03c1\u03ac\u03b3\u03b5\u03b9 \u03c4\u03b7\u03bd \u03b1\u03bb\u03bb\u03b1\u03b3\u03ae.<\/p>\n<\/li>\n<li><span class=\"td-step-kicker\">\u0421\u0442\u044a\u043f\u043a\u0430 5<\/span><strong>\u0391\u03bd\u03b1\u03bb\u03cd\u03c3\u03c4\u03b5 \u03ba\u03ac\u03b8\u03b5 benchmark \u03b1\u03bd\u03ac slice<\/strong>\n<p>\u0395\u03bb\u03ad\u03b3\u03be\u03c4\u03b5 base rate, answer format, near-chance subsets \u03ba\u03b1\u03b9 correlation \u03bc\u03b5 cross-domain \u03ae production-like \u03b4\u03b5\u03b4\u03bf\u03bc\u03ad\u03bd\u03b1 \u03c0\u03c1\u03b9\u03bd \u03b5\u03bc\u03c0\u03b9\u03c3\u03c4\u03b5\u03c5\u03c4\u03b5\u03af\u03c4\u03b5 pooled score.<\/p>\n<\/li>\n<li><span class=\"td-step-kicker\">\u0421\u0442\u044a\u043f\u043a\u0430 6<\/span><strong>\u03a0\u03c1\u03bf\u03ba\u03b1\u03b8\u03bf\u03c1\u03af\u03c3\u03c4\u03b5 gates \u03ba\u03b1\u03b9 rollback<\/strong>\n<p>\u039f\u03c1\u03af\u03c3\u03c4\u03b5 thresholds, blind protocol, multi-seed replication \u03cc\u03c0\u03bf\u03c5 \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03ba\u03b1\u03b9 \u03c4\u03b5\u03ba\u03bc\u03b7\u03c1\u03b9\u03c9\u03bc\u03ad\u03bd\u03b7 \u03b4\u03b9\u03b1\u03b4\u03b9\u03ba\u03b1\u03c3\u03af\u03b1 \u03b1\u03c0\u03cc\u03c3\u03c5\u03c1\u03c3\u03b7\u03c2 \u03c3\u03c5\u03bc\u03c0\u03b5\u03c1\u03ac\u03c3\u03bc\u03b1\u03c4\u03bf\u03c2 \u03cc\u03c4\u03b1\u03bd \u03bf scorer \u03ae \u03c4\u03bf dataset \u03b1\u03c0\u03bf\u03b4\u03b5\u03b9\u03c7\u03b8\u03b5\u03af \u03b5\u03bb\u03b1\u03c4\u03c4\u03c9\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc.<\/p>\n<\/li>\n<\/ol>\n<\/div>\n<p>\u0397 \u03c0\u03c1\u03b1\u03ba\u03c4\u03b9\u03ba\u03ae \u03b1\u03be\u03af\u03b1 \u03b1\u03c5\u03c4\u03ce\u03bd \u03c4\u03c9\u03bd \u03b5\u03bb\u03ad\u03b3\u03c7\u03c9\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03cc\u03c4\u03b9 \u03bc\u03b5\u03c4\u03b1\u03c4\u03c1\u03ad\u03c0\u03bf\u03c5\u03bd \u03c4\u03b7\u03bd \u03b5\u03ba\u03c0\u03b1\u03af\u03b4\u03b5\u03c5\u03c3\u03b7 \u03b1\u03c0\u03cc \u03b1\u03b4\u03b9\u03b1\u03c6\u03b1\u03bd\u03ae \u03b4\u03b1\u03c0\u03ac\u03bd\u03b7 compute \u03c3\u03b5 \u03b5\u03bb\u03b5\u03b3\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf \u03c3\u03cd\u03c3\u03c4\u03b7\u03bc\u03b1 \u03b1\u03c0\u03bf\u03c6\u03ac\u03c3\u03b5\u03c9\u03bd. \u0397 \u03b1\u03bd\u03ac\u03bb\u03c5\u03c3\u03b7 <a href=\"https:\/\/twodots.gr\/actor-judge-editor-ai-schediasmos-choris-allagi-stochou\/\">Actor, Judge, Editor<\/a> \u03b4\u03b5\u03af\u03c7\u03bd\u03b5\u03b9 \u03bc\u03b9\u03b1 \u03c3\u03c5\u03b3\u03b3\u03b5\u03bd\u03ae \u03b1\u03c1\u03c7\u03ae \u03c3\u03c4\u03b7\u03bd \u03b5\u03ba\u03c4\u03ad\u03bb\u03b5\u03c3\u03b7: \u03cc\u03c4\u03b1\u03bd \u03b7 \u03b1\u03be\u03b9\u03bf\u03bb\u03cc\u03b3\u03b7\u03c3\u03b7 \u03ba\u03b1\u03b9 \u03b7 \u03b5\u03c0\u03b9\u03c3\u03ba\u03b5\u03c5\u03ae \u03ad\u03c7\u03bf\u03c5\u03bd \u03c3\u03b1\u03c6\u03ae \u03cc\u03c1\u03b9\u03b1, \u03b7 \u03b2\u03b5\u03bb\u03c4\u03af\u03c9\u03c3\u03b7 \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03c0\u03b1\u03c1\u03b1\u03bc\u03ad\u03bd\u03b5\u03b9 \u03c0\u03b9\u03c3\u03c4\u03ae \u03c3\u03c4\u03bf\u03bd \u03b1\u03c1\u03c7\u03b9\u03ba\u03cc \u03c3\u03c4\u03cc\u03c7\u03bf.<\/p>\n<h2 id=\"limitations-conclusion\">\u03a0\u03b5\u03c1\u03b9\u03bf\u03c1\u03b9\u03c3\u03bc\u03bf\u03af \u03ba\u03b1\u03b9 \u03c3\u03c9\u03c3\u03c4\u03cc \u03c3\u03c5\u03bc\u03c0\u03ad\u03c1\u03b1\u03c3\u03bc\u03b1<\/h2>\n<p>\u038c\u03bb\u03b1 \u03c4\u03b1 arms \u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b1\u03bd seed 42. \u039f\u03b9 \u03c3\u03c5\u03b3\u03b3\u03c1\u03b1\u03c6\u03b5\u03af\u03c2 \u03b5\u03ba\u03c4\u03b9\u03bc\u03bf\u03cd\u03bd \u03b1\u03c0\u03cc \u03c0\u03c1\u03bf\u03b7\u03b3\u03bf\u03cd\u03bc\u03b5\u03bd\u03b1 runs \u03cc\u03c4\u03b9 \u03b7 seed-to-seed \u03c4\u03c5\u03c0\u03b9\u03ba\u03ae \u03b1\u03c0\u03cc\u03ba\u03bb\u03b9\u03c3\u03b7 \u03c4\u03bf\u03c5 GRPO \u03b5\u03af\u03bd\u03b1\u03b9 \u03c0\u03b5\u03c1\u03af\u03c0\u03bf\u03c5 2,4 \u03ad\u03c9\u03c2 2,7 \u03bc\u03bf\u03bd\u03ac\u03b4\u03b5\u03c2. \u039f\u03b9 \u03bc\u03b5\u03b3\u03b1\u03bb\u03cd\u03c4\u03b5\u03c1\u03b5\u03c2 \u03bc\u03b5\u03c4\u03b1\u03be\u03cd-group \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03ad\u03c2 \u03be\u03b5\u03c0\u03b5\u03c1\u03bd\u03bf\u03cd\u03bd \u03b1\u03c5\u03c4\u03cc \u03c4\u03bf \u03b5\u03cd\u03c1\u03bf\u03c2, \u03b1\u03bb\u03bb\u03ac \u03bf\u03bb\u03cc\u03ba\u03bb\u03b7\u03c1\u03b7 \u03b7 \u03c3\u03b5\u03b9\u03c1\u03ac \u03bc\u03ad\u03c3\u03b1 \u03c3\u03c4\u03b7\u03bd delivered-prior \u03bf\u03bc\u03ac\u03b4\u03b1 \u03b4\u03b5\u03bd \u03c0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03bc\u03b5\u03c4\u03b1\u03c4\u03c1\u03b1\u03c0\u03b5\u03af \u03c3\u03b5 \u03c3\u03c4\u03b1\u03b8\u03b5\u03c1\u03cc ranking \u03c7\u03c9\u03c1\u03af\u03c2 multi-seed replication.<\/p>\n<p>\u03a4\u03bf pooled MathV metric \u03b5\u03af\u03bd\u03b1\u03b9 size-weighted summary \u03b4\u03cd\u03bf slices \u03c0\u03bf\u03c5 \u03b4\u03b5\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03c4\u03c5\u03c7\u03b1\u03af\u03b1 \u03b4\u03b5\u03af\u03b3\u03bc\u03b1\u03c4\u03b1 \u03c4\u03bf\u03c5 \u03af\u03b4\u03b9\u03bf\u03c5 \u03c0\u03bb\u03b7\u03b8\u03c5\u03c3\u03bc\u03bf\u03cd. \u03a4\u03bf sparse-800 \u03bc\u03bf\u03b9\u03c1\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 distribution \u03ba\u03b1\u03b9 template families \u03bc\u03b5 \u03c4\u03bf training pool, \u03b5\u03bd\u03ce \u03c0\u03b5\u03c1\u03af\u03c0\u03bf\u03c5 3 \u03bc\u03bf\u03bd\u03ac\u03b4\u03b5\u03c2 \u03c4\u03bf\u03c5 \u03b9\u03c3\u03c7\u03c5\u03c1\u03cc\u03c4\u03b5\u03c1\u03bf\u03c5 arm \u03b1\u03c0\u03bf\u03b4\u03af\u03b4\u03bf\u03bd\u03c4\u03b1\u03b9 \u03c3\u03b5 template duplicates. \u03a4\u03bf DynaMath \u03c0\u03b1\u03c1\u03ad\u03c7\u03b5\u03b9 \u03c4\u03bf \u03b2\u03b1\u03c3\u03b9\u03ba\u03cc cross-domain evidence, \u03b1\u03bb\u03bb\u03ac \u03c4\u03bf hard split \u03ad\u03c7\u03b5\u03b9 \u03bc\u03cc\u03bd\u03bf 104 \u03b5\u03c1\u03c9\u03c4\u03ae\u03c3\u03b5\u03b9\u03c2 \u03ba\u03b1\u03b9 \u03b5\u03af\u03bd\u03b1\u03b9 \u03c5\u03c0\u03bf\u03c4\u03c1\u03bf\u03c6\u03bf\u03b4\u03bf\u03c4\u03b7\u03bc\u03ad\u03bd\u03bf \u03b3\u03b9\u03b1 \u03bb\u03b5\u03c0\u03c4\u03ad\u03c2 \u03c3\u03c5\u03b3\u03ba\u03c1\u03af\u03c3\u03b5\u03b9\u03c2.<\/p>\n<p>\u03a4\u03b1 supervision regimes \u03b5\u03c0\u03af\u03c3\u03b7\u03c2 \u03b4\u03b9\u03b1\u03c6\u03ad\u03c1\u03bf\u03c5\u03bd: \u03c4\u03b1 distillation arms \u03b4\u03b5\u03bd \u03b2\u03bb\u03ad\u03c0\u03bf\u03c5\u03bd ground-truth answers, \u03b5\u03bd\u03ce \u03c4\u03b1 reward-driven arms \u03c4\u03b1 \u03c7\u03c1\u03b7\u03c3\u03b9\u03bc\u03bf\u03c0\u03bf\u03b9\u03bf\u03cd\u03bd. \u039f LLM judge \u03ad\u03c7\u03b5\u03b9 \u03bc\u03b9\u03ba\u03c1\u03ae \u03b1\u03bb\u03bb\u03ac \u03c5\u03c0\u03b1\u03c1\u03ba\u03c4\u03ae \u03b4\u03b9\u03b1\u03ba\u03cd\u03bc\u03b1\u03bd\u03c3\u03b7. \u03a4\u03b1 hint-conditioned rollouts \u03b4\u03b5\u03bd \u03ad\u03c7\u03bf\u03c5\u03bd \u03b1\u03ba\u03cc\u03bc\u03b7 off-context importance correction. \u0391\u03c5\u03c4\u03bf\u03af \u03bf\u03b9 \u03c0\u03b5\u03c1\u03b9\u03bf\u03c1\u03b9\u03c3\u03bc\u03bf\u03af \u03b5\u03bc\u03c0\u03bf\u03b4\u03af\u03b6\u03bf\u03c5\u03bd \u03c4\u03b7\u03bd \u03b1\u03bd\u03b1\u03b3\u03c9\u03b3\u03ae \u03c4\u03c9\u03bd \u03c3\u03c5\u03b3\u03ba\u03b5\u03ba\u03c1\u03b9\u03bc\u03ad\u03bd\u03c9\u03bd scores \u03c3\u03b5 \u03ba\u03b1\u03b8\u03bf\u03bb\u03b9\u03ba\u03ae \u03c3\u03c5\u03bd\u03c4\u03b1\u03b3\u03ae.<\/p>\n<p>\u03a4\u03bf \u03c3\u03c9\u03c3\u03c4\u03cc \u03c3\u03c5\u03bc\u03c0\u03ad\u03c1\u03b1\u03c3\u03bc\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 \u03c3\u03c4\u03b5\u03bd\u03cc\u03c4\u03b5\u03c1\u03bf \u03ba\u03b1\u03b9 \u03c0\u03b9\u03bf \u03c7\u03c1\u03ae\u03c3\u03b9\u03bc\u03bf. \u03a3\u03b5 \u03ad\u03bd\u03b1 \u03b5\u03bb\u03b5\u03b3\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf sparse-reward visual-math experiment, delivered priors \u03be\u03b5\u03c7\u03ce\u03c1\u03b9\u03c3\u03b1\u03bd \u03b1\u03c0\u03cc priors \u03c0\u03bf\u03c5 \u03c7\u03ac\u03bd\u03bf\u03bd\u03c4\u03b1\u03bd \u03ae \u03c0\u03b5\u03c1\u03b9\u03bf\u03c1\u03af\u03b6\u03bf\u03bd\u03c4\u03b1\u03bd, \u03c4\u03b1 hints \u03b2\u03bf\u03ae\u03b8\u03b7\u03c3\u03b1\u03bd \u03ba\u03c5\u03c1\u03af\u03c9\u03c2 \u03c4\u03b7\u03bd exploration, \u03c4\u03bf HL-Gauss \u03ac\u03bb\u03bb\u03b1\u03be\u03b5 \u03b4\u03c1\u03b1\u03c3\u03c4\u03b9\u03ba\u03ac \u03c4\u03b7\u03bd \u03b5\u03b9\u03ba\u03cc\u03bd\u03b1 \u03c4\u03bf\u03c5 critic \u03ba\u03b1\u03b9 \u03ad\u03bd\u03b1 \u03c6\u03b1\u03b9\u03bd\u03bf\u03bc\u03b5\u03bd\u03b9\u03ba\u03ac \u03b3\u03b5\u03bd\u03b9\u03ba\u03cc benchmark \u03b1\u03bd\u03c4\u03ad\u03c3\u03c4\u03c1\u03b5\u03c8\u03b5 \u03c4\u03b7 \u03c3\u03c7\u03ad\u03c3\u03b7 \u03bc\u03b5 \u03c4\u03bf \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc cross-domain test.<\/p>\n<p>\u0393\u03b9\u03b1 product \u03ba\u03b1\u03b9 \u03b4\u03b9\u03bf\u03af\u03ba\u03b7\u03c3\u03b7, \u03b7 \u03b5\u03c1\u03ce\u03c4\u03b7\u03c3\u03b7 \u03b4\u03b5\u03bd \u03b5\u03af\u03bd\u03b1\u03b9 \u03b1\u03c0\u03bb\u03ce\u03c2 \u00ab\u03c0\u03bf\u03b9\u03bf \u03bc\u03bf\u03bd\u03c4\u03ad\u03bb\u03bf \u03ba\u03ad\u03c1\u03b4\u03b9\u03c3\u03b5;\u00bb. \u0395\u03af\u03bd\u03b1\u03b9 \u03b1\u03bd \u03c4\u03bf learning signal \u03c6\u03c4\u03ac\u03bd\u03b5\u03b9 \u03c3\u03c4\u03b7\u03bd policy, \u03b1\u03bd \u03bf scorer \u03bc\u03b5\u03c4\u03c1\u03ac \u03c4\u03b7 \u03c3\u03c9\u03c3\u03c4\u03ae \u03c3\u03c5\u03bc\u03c0\u03b5\u03c1\u03b9\u03c6\u03bf\u03c1\u03ac \u03ba\u03b1\u03b9 \u03b1\u03bd \u03c4\u03bf test set \u03c0\u03c1\u03bf\u03b2\u03bb\u03ad\u03c0\u03b5\u03b9 \u03c4\u03b7\u03bd \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03ae \u03c7\u03c1\u03ae\u03c3\u03b7. \u03a7\u03c9\u03c1\u03af\u03c2 \u03b1\u03c5\u03c4\u03ad\u03c2 \u03c4\u03b9\u03c2 \u03c4\u03c1\u03b5\u03b9\u03c2 \u03b1\u03c0\u03bf\u03b4\u03b5\u03af\u03be\u03b5\u03b9\u03c2, \u03ad\u03bd\u03b1 \u03c5\u03c8\u03b7\u03bb\u03cc\u03c4\u03b5\u03c1\u03bf score \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b5\u03af\u03bd\u03b1\u03b9 \u03b1\u03ba\u03c1\u03b9\u03b2\u03ae\u03c2 \u03bc\u03ad\u03c4\u03c1\u03b7\u03c3\u03b7 \u03c4\u03bf\u03c5 \u03bb\u03ac\u03b8\u03bf\u03c2 \u03c0\u03c1\u03ac\u03b3\u03bc\u03b1\u03c4\u03bf\u03c2.<\/p>\n<section class=\"td-service-cta\">\n<div class=\"td-service-cta-content\">\n<p class=\"td-service-cta-kicker\">\u0391\u03c0\u03cc \u03c4\u03bf \u03c0\u03b5\u03af\u03c1\u03b1\u03bc\u03b1 \u03c3\u03b5 \u03b5\u03bb\u03b5\u03b3\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf AI workflow<\/p>\n<p class=\"td-service-cta-title\">\u03a3\u03c7\u03b5\u03b4\u03b9\u03ac\u03c3\u03c4\u03b5 evaluation, observability \u03ba\u03b1\u03b9 rollout gates \u03c0\u03c1\u03b9\u03bd \u03ba\u03bb\u03b9\u03bc\u03b1\u03ba\u03ce\u03c3\u03b5\u03c4\u03b5 \u03c4\u03bf\u03bd agent<\/p>\n<p>\u0397 TWO DOTS \u03c7\u03b1\u03c1\u03c4\u03bf\u03b3\u03c1\u03b1\u03c6\u03b5\u03af rewards, failure slices, scorer contracts, teacher \u03ae verifier signals, acceptance tests \u03ba\u03b1\u03b9 monitoring \u03ce\u03c3\u03c4\u03b5 \u03b7 \u03b2\u03b5\u03bb\u03c4\u03af\u03c9\u03c3\u03b7 \u03bd\u03b1 \u03bc\u03b5\u03c4\u03c1\u03b9\u03ad\u03c4\u03b1\u03b9 \u03c3\u03c4\u03bf \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03cc \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03b7\u03c3\u03b9\u03b1\u03ba\u03cc \u03b1\u03c0\u03bf\u03c4\u03ad\u03bb\u03b5\u03c3\u03bc\u03b1 \u2014 \u03cc\u03c7\u03b9 \u03bc\u03cc\u03bd\u03bf \u03c3\u03b5 \u03ad\u03bd\u03b1 \u03b2\u03bf\u03bb\u03b9\u03ba\u03cc benchmark.<\/p>\n<div class=\"td-service-cta-actions\"><a class=\"td-service-cta-button\" href=\"https:\/\/twodots.gr\/aftomatismoi-epicheiriseon-ai\/\">\u03a3\u03c5\u03b6\u03b7\u03c4\u03ae\u03c3\u03c4\u03b5 \u03ad\u03bd\u03b1\u03bd \u03b5\u03bb\u03b5\u03b3\u03c7\u03cc\u03bc\u03b5\u03bd\u03bf AI \u03b1\u03c5\u03c4\u03bf\u03bc\u03b1\u03c4\u03b9\u03c3\u03bc\u03cc<\/a><\/div>\n<\/div>\n<\/section>\n<section id=\"sychnes-erotiseis\" class=\"td-faq-section\">\n<div class=\"td-faq\">\n<p class=\"td-faq-heading\">\u0427\u0435\u0441\u0442\u043e \u0437\u0430\u0434\u0430\u0432\u0430\u043d\u0438 \u0432\u044a\u043f\u0440\u043e\u0441\u0438<\/p>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u03a4\u03b9 \u03b5\u03af\u03bd\u03b1\u03b9 sparse reward \u03c3\u03c4\u03b7\u03bd \u03b5\u03ba\u03c0\u03b1\u03af\u03b4\u03b5\u03c5\u03c3\u03b7 AI;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u0395\u03af\u03bd\u03b1\u03b9 \u03ba\u03b1\u03b8\u03b5\u03c3\u03c4\u03ce\u03c2 \u03cc\u03c0\u03bf\u03c5 \u03bf\u03b9 \u03b5\u03c0\u03b9\u03c4\u03c5\u03c7\u03b7\u03bc\u03ad\u03bd\u03b5\u03c2 \u03c0\u03c1\u03bf\u03c3\u03c0\u03ac\u03b8\u03b5\u03b9\u03b5\u03c2 \u03b5\u03af\u03bd\u03b1\u03b9 \u03c3\u03c0\u03ac\u03bd\u03b9\u03b5\u03c2 \u03ba\u03b1\u03b9 \u03c0\u03bf\u03bb\u03bb\u03ac rollouts \u03c0\u03b1\u03af\u03c1\u03bd\u03bf\u03c5\u03bd \u03c4\u03bf \u03af\u03b4\u03b9\u03bf \u03bc\u03b7\u03b4\u03b5\u03bd\u03b9\u03ba\u03cc reward. \u03a3\u03b5 group-relative methods \u03b1\u03c5\u03c4\u03cc \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b1\u03c6\u03b1\u03b9\u03c1\u03b5\u03af \u03c4\u03b7 \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03bf\u03c0\u03bf\u03af\u03b7\u03c3\u03b7 \u03c0\u03bf\u03c5 \u03c7\u03c1\u03b5\u03b9\u03ac\u03b6\u03b5\u03c4\u03b1\u03b9 \u03b7 policy \u03b3\u03b9\u03b1 \u03bd\u03b1 \u03bc\u03ac\u03b8\u03b5\u03b9.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u0393\u03b9\u03b1\u03c4\u03af \u03c4\u03b1 all-wrong GRPO groups \u03b4\u03af\u03bd\u03bf\u03c5\u03bd \u03bc\u03b7\u03b4\u03b5\u03bd\u03b9\u03ba\u03cc \u03c3\u03ae\u03bc\u03b1;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u03a4\u03bf GRPO \u03c5\u03c0\u03bf\u03bb\u03bf\u03b3\u03af\u03b6\u03b5\u03b9 \u03c3\u03c7\u03b5\u03c4\u03b9\u03ba\u03cc \u03c0\u03bb\u03b5\u03bf\u03bd\u03ad\u03ba\u03c4\u03b7\u03bc\u03b1 \u03bc\u03ad\u03c3\u03b1 \u03c3\u03c4\u03b7\u03bd \u03bf\u03bc\u03ac\u03b4\u03b1. \u038c\u03c4\u03b1\u03bd \u03cc\u03bb\u03b1 \u03c4\u03b1 rollouts \u03ad\u03c7\u03bf\u03c5\u03bd \u03c4\u03bf \u03af\u03b4\u03b9\u03bf reward, \u03bf\u03b9 group-normalized advantages \u03b1\u03c0\u03cc \u03c4\u03bf task reward \u03bc\u03b7\u03b4\u03b5\u03bd\u03af\u03b6\u03bf\u03bd\u03c4\u03b1\u03b9 \u03ba\u03b1\u03b9 \u03b4\u03b5\u03bd \u03ba\u03b1\u03c4\u03b5\u03c5\u03b8\u03cd\u03bd\u03bf\u03c5\u03bd \u03c4\u03b7\u03bd \u03b5\u03bd\u03b7\u03bc\u03ad\u03c1\u03c9\u03c3\u03b7 \u03c4\u03b7\u03c2 policy.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u03a0\u03bf\u03b9\u03bf\u03b9 \u03ae\u03c4\u03b1\u03bd \u03bf\u03b9 \u03c4\u03c1\u03b5\u03b9\u03c2 \u03c4\u03cd\u03c0\u03bf\u03b9 prior;<\/summary>\n<div class=\"td-faq-content\">\n<p>Text prior \u03bc\u03ad\u03c3\u03c9 reference-solution hints, distribution prior \u03bc\u03ad\u03c3\u03c9 on-policy distillation \u03b1\u03c0\u03cc 7B teacher \u03ba\u03b1\u03b9 value prior \u03bc\u03ad\u03c3\u03c9 pretrained critic.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u03a0\u03bf\u03b9\u03b1 \u03bc\u03ad\u03b8\u03bf\u03b4\u03bf\u03c2 \u03b5\u03af\u03c7\u03b5 \u03c4\u03bf \u03c5\u03c8\u03b7\u03bb\u03cc\u03c4\u03b5\u03c1\u03bf DynaMath score;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u03a3\u03c4\u03b7 \u03c3\u03c5\u03b3\u03ba\u03b5\u03ba\u03c1\u03b9\u03bc\u03ad\u03bd\u03b7 \u03b5\u03ba\u03c4\u03ad\u03bb\u03b5\u03c3\u03b7 \u03c4\u03bf uft002 \u03ad\u03c6\u03c4\u03b1\u03c3\u03b5 32,9%. \u039f \u03ad\u03bd\u03b1\u03c2 training seed, \u03c4\u03bf \u03bc\u03ad\u03b3\u03b5\u03b8\u03bf\u03c2 \u03c4\u03bf\u03c5 set \u03ba\u03b1\u03b9 \u03b7 judge variance \u03b4\u03b5\u03bd \u03b5\u03c0\u03b9\u03c4\u03c1\u03ad\u03c0\u03bf\u03c5\u03bd \u03bd\u03b1 \u03b8\u03b5\u03c9\u03c1\u03b7\u03b8\u03b5\u03af \u03b7 \u03bc\u03b9\u03ba\u03c1\u03ae \u03b4\u03b9\u03b1\u03c6\u03bf\u03c1\u03ac \u03bc\u03ad\u03c3\u03b1 \u03c3\u03c4\u03b7\u03bd \u03ba\u03b1\u03bb\u03cd\u03c4\u03b5\u03c1\u03b7 \u03bf\u03bc\u03ac\u03b4\u03b1 \u03c3\u03c4\u03b1\u03b8\u03b5\u03c1\u03ae \u03ba\u03b1\u03c4\u03ac\u03c4\u03b1\u03be\u03b7.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u0393\u03b9\u03b1\u03c4\u03af \u03ae\u03c4\u03b1\u03bd \u03c3\u03b7\u03bc\u03b1\u03bd\u03c4\u03b9\u03ba\u03cc \u03c4\u03bf HL-Gauss critic loss;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u0397 \u03b1\u03bd\u03c4\u03b9\u03ba\u03b1\u03c4\u03ac\u03c3\u03c4\u03b1\u03c3\u03b7 \u03c4\u03bf\u03c5 clipped-MSE value loss \u03bc\u03b5 categorical HL-Gauss \u03c3\u03c5\u03bd\u03b4\u03ad\u03b8\u03b7\u03ba\u03b5 \u03bc\u03b5 \u03ac\u03bd\u03bf\u03b4\u03bf \u03b1\u03c0\u03cc 39,2% \u03c3\u03b5 53,6% \u03c3\u03c4\u03bf sparse-800 \u03ba\u03b1\u03b9 \u03b1\u03c0\u03cc 27,0% \u03c3\u03b5 31,9% \u03c3\u03c4\u03bf DynaMath, \u03bc\u03b5 \u03cc\u03bb\u03b1 \u03c4\u03b1 \u03ac\u03bb\u03bb\u03b1 \u03b2\u03b1\u03c3\u03b9\u03ba\u03ac \u03c3\u03c4\u03bf\u03b9\u03c7\u03b5\u03af\u03b1 \u03af\u03b4\u03b9\u03b1.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u0393\u03b9\u03b1\u03c4\u03af \u03b1\u03c0\u03ad\u03c4\u03c5\u03c7\u03b5 \u03c4\u03bf old-319 \u03c9\u03c2 generalization check;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u03a4\u03bf 69% \u03c4\u03bf\u03c5 set \u03ae\u03c4\u03b1\u03bd \u03c4\u03b5\u03c4\u03c1\u03b1\u03c0\u03bb\u03ae\u03c2 \u03b5\u03c0\u03b9\u03bb\u03bf\u03b3\u03ae\u03c2 \u03ba\u03b1\u03b9 \u03cc\u03bb\u03b1 \u03c4\u03b1 arms \u03ba\u03b9\u03bd\u03bf\u03cd\u03bd\u03c4\u03b1\u03bd \u03ba\u03bf\u03bd\u03c4\u03ac \u03c3\u03c4\u03bf \u03c4\u03c5\u03c7\u03b1\u03af\u03bf 25%. \u03a4\u03bf slice \u03c6\u03b1\u03b9\u03bd\u03cc\u03c4\u03b1\u03bd \u03bd\u03b1 \u03b5\u03c0\u03b9\u03b2\u03c1\u03b1\u03b2\u03b5\u03cd\u03b5\u03b9 \u03c3\u03c5\u03bc\u03c0\u03b5\u03c1\u03b9\u03c6\u03bf\u03c1\u03ac \u03ba\u03bf\u03bd\u03c4\u03ac \u03c3\u03c4\u03bf base model \u03ba\u03b1\u03b9 \u03b5\u03af\u03c7\u03b5 \u03b1\u03c1\u03bd\u03b7\u03c4\u03b9\u03ba\u03ae \u03c3\u03c5\u03c3\u03c7\u03ad\u03c4\u03b9\u03c3\u03b7 \u22120,74 \u03bc\u03b5 \u03c4\u03bf DynaMath.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u039c\u03c0\u03bf\u03c1\u03bf\u03cd\u03bd \u03b4\u03cd\u03bf priors \u03bd\u03b1 \u03b4\u03ce\u03c3\u03bf\u03c5\u03bd \u03b4\u03b9\u03c0\u03bb\u03cc \u03cc\u03c6\u03b5\u03bb\u03bf\u03c2;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u038c\u03c7\u03b9 \u03b1\u03c0\u03b1\u03c1\u03b1\u03af\u03c4\u03b7\u03c4\u03b1. \u03a3\u03c4\u03bf hintsaf, hint \u03ba\u03b1\u03b9 annealed teacher signal \u03b4\u03b5\u03bd \u03c0\u03c1\u03cc\u03c3\u03b8\u03b5\u03c3\u03b1\u03bd \u03ba\u03b1\u03b8\u03b1\u03c1\u03ae \u03c5\u03c0\u03b5\u03c1\u03b1\u03be\u03af\u03b1, \u03c0\u03b9\u03b8\u03b1\u03bd\u03ce\u03c2 \u03b5\u03c0\u03b5\u03b9\u03b4\u03ae \u03b1\u03bd\u03c4\u03b9\u03bc\u03b5\u03c4\u03ce\u03c0\u03b9\u03b6\u03b1\u03bd \u03c4\u03bf \u03af\u03b4\u03b9\u03bf bottleneck \u03c4\u03c9\u03bd all-wrong groups.<\/p>\n<\/div>\n<\/details>\n<details class=\"td-faq-item\">\n<summary class=\"td-faq-title\">\u03a0\u03bf\u03b9\u03bf \u03b5\u03af\u03bd\u03b1\u03b9 \u03c4\u03bf \u03b2\u03b1\u03c3\u03b9\u03ba\u03cc \u03bc\u03ac\u03b8\u03b7\u03bc\u03b1 \u03b3\u03b9\u03b1 \u03b5\u03c0\u03b9\u03c7\u03b5\u03b9\u03c1\u03ae\u03c3\u03b5\u03b9\u03c2;<\/summary>\n<div class=\"td-faq-content\">\n<p>\u03a0\u03c1\u03ad\u03c0\u03b5\u03b9 \u03bd\u03b1 \u03b5\u03c0\u03b1\u03bb\u03b7\u03b8\u03b5\u03cd\u03b5\u03c4\u03b1\u03b9 \u03c4\u03cc\u03c3\u03bf \u03b7 \u03c0\u03c1\u03b1\u03b3\u03bc\u03b1\u03c4\u03b9\u03ba\u03ae \u03c0\u03b1\u03c1\u03ac\u03b4\u03bf\u03c3\u03b7 \u03c4\u03bf\u03c5 learning signal \u03cc\u03c3\u03bf \u03ba\u03b1\u03b9 \u03b7 \u03b5\u03b3\u03ba\u03c5\u03c1\u03cc\u03c4\u03b7\u03c4\u03b1 \u03c4\u03bf\u03c5 measurement pipeline. \u0388\u03bd\u03b1 \u03b5\u03bd\u03b5\u03c1\u03b3\u03cc feature \u03ba\u03b1\u03b9 \u03ad\u03bd\u03b1 held-out dataset \u03b4\u03b5\u03bd \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03bf\u03cd\u03bd \u03b1\u03c0\u03cc \u03bc\u03cc\u03bd\u03b1 \u03c4\u03bf\u03c5\u03c2 \u03b1\u03c0\u03cc\u03b4\u03b5\u03b9\u03be\u03b7 \u03b2\u03b5\u03bb\u03c4\u03af\u03c9\u03c3\u03b7\u03c2.<\/p>\n<\/div>\n<\/details>\n<\/div>\n<\/section>\n<div class=\"td-source-list\">\n<p id=\"piges\" class=\"td-source-list-title\">\u0418\u0437\u0442\u043e\u0447\u043d\u0438\u0446\u0438<\/p>\n<ul>\n<li><a href=\"https:\/\/arxiv.org\/abs\/2608.21811\" target=\"_blank\" rel=\"noopener\">arXiv \u2014 Hints, Critics, and Teachers: Prior Injection for Sparse-Reward RL in Vision-Language Math Reasoning<\/a><\/li>\n<li><a href=\"https:\/\/arxiv.org\/html\/2608.21811\" target=\"_blank\" rel=\"noopener\">arXiv HTML \u2014 \u03c0\u03bb\u03ae\u03c1\u03b7\u03c2 \u03bc\u03b5\u03b8\u03bf\u03b4\u03bf\u03bb\u03bf\u03b3\u03af\u03b1, \u03b1\u03c0\u03bf\u03c4\u03b5\u03bb\u03ad\u03c3\u03bc\u03b1\u03c4\u03b1, audit \u03ba\u03b1\u03b9 \u03c0\u03b5\u03c1\u03b9\u03bf\u03c1\u03b9\u03c3\u03bc\u03bf\u03af \u03c4\u03b7\u03c2 \u03c3\u03cd\u03b3\u03ba\u03c1\u03b9\u03c3\u03b7\u03c2<\/a><\/li>\n<li><a href=\"https:\/\/arxiv.org\/abs\/2402.03300\" target=\"_blank\" rel=\"noopener\">arXiv \u2014 DeepSeekMath \u03ba\u03b1\u03b9 \u03b7 \u03b5\u03b9\u03c3\u03b1\u03b3\u03c9\u03b3\u03ae \u03c4\u03bf\u03c5 Group Relative Policy Optimization<\/a><\/li>\n<li><a href=\"https:\/\/arxiv.org\/abs\/2411.00836\" target=\"_blank\" rel=\"noopener\">arXiv \u2014 DynaMath: dynamic visual benchmark \u03b3\u03b9\u03b1 robustness \u03bc\u03b1\u03b8\u03b7\u03bc\u03b1\u03c4\u03b9\u03ba\u03bf\u03cd reasoning<\/a><\/li>\n<li><a href=\"https:\/\/arxiv.org\/abs\/2608.02181\" target=\"_blank\" rel=\"noopener\">arXiv \u2014 Start Classifying: Categorical Critics for LLM Reinforcement Learning<\/a><\/li>\n<li><a href=\"https:\/\/arxiv.org\/abs\/2402.01781\" target=\"_blank\" rel=\"noopener\">arXiv \u2014 When Benchmarks are Targets: \u03b5\u03c5\u03b1\u03b9\u03c3\u03b8\u03b7\u03c3\u03af\u03b1 \u03c4\u03c9\u03bd LLM leaderboards<\/a><\/li>\n<\/ul>\n<\/div>","protected":false},"excerpt":{"rendered":"<p>\u039c\u03b9\u03b1 \u03b5\u03bb\u03b5\u03b3\u03c7\u03cc\u03bc\u03b5\u03bd\u03b7 \u03c3\u03cd\u03b3\u03ba\u03c1\u03b9\u03c3\u03b7 11 \u03bc\u03b5\u03b8\u03cc\u03b4\u03c9\u03bd \u03b4\u03b5\u03af\u03c7\u03bd\u03b5\u03b9 \u03c0\u03cc\u03c4\u03b5 hints, distillation \u03ba\u03b1\u03b9 critics \u03b2\u03bf\u03b7\u03b8\u03bf\u03cd\u03bd \u03c4\u03b7\u03bd AI \u03bc\u03b5 sparse reward \u2014 \u03ba\u03b1\u03b9 \u03c0\u03ce\u03c2 \u03ad\u03bd\u03b1 benchmark \u03bc\u03c0\u03bf\u03c1\u03b5\u03af \u03bd\u03b1 \u03b1\u03bd\u03c4\u03b9\u03c3\u03c4\u03c1\u03ad\u03c8\u03b5\u03b9 \u03c4\u03bf \u03c3\u03c5\u03bc\u03c0\u03ad\u03c1\u03b1\u03c3\u03bc\u03b1.<\/p>","protected":false},"author":1,"featured_media":98875,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"_gspb_post_css":"","content-type":"","footnotes":""},"categories":[199],"tags":[18888,7455,19313,18034,20630],"class_list":["post-97814","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-techniti-noimosyni","tag-ai-evaluation","tag-ai-training","tag-grpo","tag-reinforcement-learning","tag-vision-language-ai"],"blocksy_meta":{"styles_descriptor":{"styles":{"desktop":"","tablet":"","mobile":""},"google_fonts":[],"version":8}},"_links":{"self":[{"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/posts\/97814","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/comments?post=97814"}],"version-history":[{"count":1,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/posts\/97814\/revisions"}],"predecessor-version":[{"id":98876,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/posts\/97814\/revisions\/98876"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/media\/98875"}],"wp:attachment":[{"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/media?parent=97814"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/categories?post=97814"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/twodots.gr\/bg\/wp-json\/wp\/v2\/tags?post=97814"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}