With over 20 years of experience, we transform your digital presence. We specialize in website and E-Shop development, SEO and Digital Marketing, ERP software and smart automation that take your business to the next level.
CoreSec: Why a Reliable AI System Must Know When Not to Provide an Answer
CoreSec explains why abstention, clearly defined roles, evidence, and an audit trail make an AI system more secure when data is missing or inconsistent.
Safe automation stops when critical signals are missing or conflict.
Answer first: A reliable AI system should not turn every incomplete dataset into a definitive answer. There needs to be an explicit right to abstain, specific rules regarding which evidence is mandatory or decisive, and a safe path back to human intervention when signals are missing or conflicting.
CoreSec, a production-ready root cause analysis system for Azure Clos networks, demonstrates how this principle works at scale. It replaces weighted scoring with PAM-style composition, five topology-aware configurations, and a verifiable audit trail. The AI abstention This is not a flaw in the system; it is the mechanism that prevents an uncertain diagnosis from resulting in an incorrect automated action.
The problem is the noise, not just the lack of data
What is more dangerous for an automated system: delaying a response or confidently providing the wrong one? In the root cause analysis of a large cloud infrastructure, the second option can trigger the wrong mitigation measures, prolong an incident, and waste the team’s valuable time on fruitless investigation. CoreSec was designed around a different principle: when data is incomplete, outdated, or contradictory, the system must be able to refrain from identifying a cause.
A large data center fabric is never in a state of perfect «health.» Cables exhibit CRC errors, links flutter, optical modules degrade, and switches reboot or are upgraded in stages. The Clos architecture features many equivalent paths, allowing traffic to bypass most of these failures without causing an immediate, visible problem for the customer.
The paper states that, at any given moment, it is normal for 0.3% to 1% of links to experience loss, flaps, or degraded optics. When a workload exhibits high latency or packet loss, the system simultaneously detects multiple suspicious entities. Some are associated with the specific incident, while others are part of the steady-state background of faults. The abundance of telemetry, therefore, does not equate to clear evidence.
Gray failures further complicate the picture. They are not complete outages, but rather partial, intermittent, or probabilistic failures: an optical module may drop a small percentage of packets in only one direction, a line card may affect flows passing through a specific ECMP path, or a firmware bug may cause brief reboots that go undetected by health monitors.
The idea goes beyond networks. Every a business workflow that uses AI to automate tasks It involves some form of judgment between incomplete data and an action that entails a cost. The difference is that in a robust system, uncertainty is not masked by a confidence score.
Why weighted scoring produced convincing but inconsistent answers
The previous approach in Azure collected signals from different telemetry agents, assigned a weight to each one, and selected the entity with the highest total. The logic sounds reasonable: the most reliable signal is given greater weight. In practice, however, reliability was not a constant property of an agent; it varied depending on the failure mode.
Active probes quickly identify connection issues, but may have gaps in coverage. Device counters provide direct hardware metrics, but they contain noise and have varying update rates. Traffic-derived signals reflect actual customer impact, but require sufficient traffic. Infrastructure summaries cover broad scenarios, but are delayed. Condensing all of this into a single number eliminates the distinction between «I have no data,» «the data is conflicting,» and «a single decisive signal is enough.».
According to the paper, the weighted approach had a false positive rate of 18% to 22%. Adjusting the weights to improve performance on one type of fault worsened performance on another, while the system was always required to identify a culprit even when the network was not the cause. This is a crucial lesson for business AI as well: the highest score among weak options does not automatically translate into a safe decision.
Weighted score
All signals are reduced to a single number. Absence, disagreement, and decisive indication lose their distinct meanings.
Forced rankingUnstable tuning
Role-based composition
Each agent is designated as requisite, required, sufficient, or optional for that specific failure surface.
Explicit roleFreshness Gate
Abstention with trace
When critical evidence is missing or signals conflict, mitigation stops, and the operator investigates why.
IndeterminateHuman escalation
Four roles for each signal
CoreSec treats RCA as a composition problem and borrows the logic of Pluggable Authentication Modules. Instead of simply asking, «How trustworthy is this agent?», it defines exactly what role it is allowed to play in the decision regarding a specific failure surface.
Requirement: If the critical signal is missing, is stale, or fails to meet its threshold, that particular configuration is immediately disabled.
Required: All required agents must pass in order for a vote to be cast; if any one fails, the configuration is rejected in the end.
Sufficient: A positive and strong signal is enough to cast a vote immediately, without waiting for the others.
Optional: It adds supporting context, but it cannot determine the outcome on its own.
For a server-to-TOR cable, the loss-of-carrier signal from the NIC can serve as a requisite, an RDMA timeout as a sufficient condition, the state of the CRC counters as a required condition, and a recent control-plane event as an optional condition. This classification encodes operational knowledge. It does not claim that a counter or an event is generally good or bad; it indicates in which failure mode and under what conditions it is decisive.
This pattern is useful for AI agents, provided it is applied as a principle rather than a ready-made formula. In a refund workflow, for example, a confirmed payment may be a requisite, an explicit cancellation by the customer may be required, and a sentiment signal may be optional. The same logic of converting instructions into verifiable AI skills It requires versioned rules, clear permissions, and human approval for sensitive actions.
The third category: healthy, unhealthy, or indeterminate
Each agent evaluates a signal as "healthy," "unhealthy," or "abstain." For noisy metrics, two thresholds are used: below the lower threshold, the metric is considered healthy; above the upper threshold, it is considered problematic; and between the two lies the abstain zone. A transient spike does not automatically result in a diagnosis.
At the merge level, CoreSec uses three states: H for healthy, U for unhealthy, and I for indeterminate. A reliable U takes precedence; H requires agreement; and a conflict results in I. The paper provides an associative merge operator so that the same set of valid evidence leads to the same result regardless of the order in which it arrives.
This distinction goes deeper than just a low confidence level. «Not enough data,» «the two systems disagree,» and «a critical prerequisite failed» can all lead to a halt, but they require different next steps. The decision trace must record the cause of the abstention, not just the final label.
The same need arises in AI workflows that maintain memory and state. One semantic rollback for AI agents It is not complete if it only restores the database but leaves the incorrect conclusion that led to the action in place. The decision status and the reason for abstention must be part of the audit trail.
Five parallel configurations instead of a single general-purpose engine
CoreSec does not attempt to solve all scenarios with a single universal formula. It runs five configurations in parallel, each corresponding to a different failure domain: server–TOR cable, TOR switch, switch–switch cable, T1 switch, and T2 switch. Each configuration has its own authoritative agents, flags, and fixed evaluation sequence.
This distinction protects the system from premature convergence. A cable fault has a different signature than a T1 failure. In the former, persistent bidirectional loss detected by active probes can be decisive. In the latter, it is necessary to correlate issues across multiple TORs and a variety of probe paths. If a configuration lacks adequate coverage, it should refrain from drawing conclusions based on unrelated signals.
The authors note that the same five configurations worked in different Azure environments without the need for recalibration for each deployment. This does not mean that every organization can simply copy these flags. It means that isolating failure surfaces and explicitly assigning roles can be more reliable than a single score that attempts to cover all cases.
How the topology selects the responsible layer
The PAM-style composition works within any configuration. Topology-aware heuristics then resolve conflicts between layers. For a TOR candidate, CoreSec uses P2.15—that is, the 97.85th percentile of the distribution of affected servers—along with a minimum requirement: at least 20% of the TOR servers must be affected.
For a T1, the rule is to consider the correlation at two-thirds of the fan-out. If at least two-thirds of the TORs connected to it indicate an unhealthy state, T1 is identified as the root cause, and the individual TOR candidates are suppressed. The sensitivity analysis in this study indicates that 50% was too lenient, 75% increased false negatives, and the range around 66% resulted in the lowest overall error.
At the cluster level, an aggregate probe drop of more than 4% triggers cluster-level attribution. Values below 3% indicated transient path instability, while values above 5% had no customer-visible impact. These specific thresholds are specific to Azure Clos fabrics and do not serve as general benchmarks for e-commerce, marketing, or customer support.
Business translation is the concept of hierarchy. A local symptom should not override a stronger system-level pattern, but a general alert should not suppress isolated faults. In a customer-support AI, for example, the word «cancellation» in a message is not sufficient grounds for a refund if the order, payment, and policy status do not match. The chatbots that support real small businesses Escalation is required when the response involves money, access, or legal terms.
Sixteen minutes for convergence without oscillations
CoreSec is activated after an incident is detected and evaluates a 16-minute telemetry window. It reruns the RCA every five minutes. The slowest agent takes about thirteen minutes, so the window leaves little room for delayed data to stabilize.
Convergence is monotonic: a lower-level attribution can be replaced by a stronger, higher-level one, but not the other way around. Once a layer’s sufficiency condition is satisfied, the decision remains stable. The paper reports that in approximately 99% of cases, the result stabilizes within the first two reruns and that the oscillations observed in the previous approach were not observed here.
If, at the end of the window, no level has sufficient data, the final response is “abstention.” Automated mitigation stops, and on-call engineers receive a structured trace: which agents did not have fresh data, which configurations were missing, which entities had partial signals, and when each source was last updated.
Waiting isn’t free. In an incident, every minute can be costly. CoreSec deliberately chooses a bounded delay to reduce the cost of a wrong action. The right balance depends on the workflow: a recommendation may allow for more time, while a security containment might require a different safe default and stricter real-time signals.
What do the production results show?
The comparison for the 2022–2025 period includes 712,345 incidents. CoreSec automatically classifies approximately 98.5%, treats 1.5% as abstentions, and reports a false positive rate of less than 1%. The previous weighted approach resulted in a false positive rate of 18% to 22%. The abstentions are explicitly counted as false negatives because the system did not provide a reason even though one existed.
The outage rate decreased from 10% in the first quarter to 1.5% in the most recent half-year, primarily due to the addition of agents that filled data gaps. The breakdown of root causes was 70% cable faults, 28% were TOR failures, and 2% were T1, T2, or cluster-level failures. Misattributions occur approximately four times a year, and incorrect mitigations occur approximately 700 times, or 0.1% of the total.
Four dimensions from the performance evaluation
The figures apply to CoreSec on Azure Clos fabrics and do not serve as a general benchmark for every AI workflow.
712.345Incidents in the 2022–2025 comparison
<1%False positives with CoreSec
1,5%abstentions in the most recent six-month period
≈99%stabilization in the first two reruns
The downstream metrics are also significant: the paper reports a decrease of over 80% in mis-triggered mitigations and 40% in customer complaints related to misattribution. The manual RCA workload—which previously required three full-time engineers—is reported to have been eliminated for incidents that are now attributed directly.
When Is Abstention a Safety Measure?
Approximately 60% of abstentions are attributed to telemetry gaps, 39% to conflicting data, and 1% to multi-cluster or datacenter-wide events. In the first two categories, abstention prevents an unsubstantiated attribution. In the third category, the problem is no longer just identifying the failing entity but coordinating multiple teams, so human judgment remains essential.
A confident but incorrect RCA forces the on-call engineer to first disprove it. An «abstention» with context guides him to the gaps in the investigation. The useful output isn’t simply the “I don’t know” label, but the decision path: which signal was stale, which configuration was selected, which abstained, and which topology rule eliminated a lower-ranking candidate.
For operational AI systems, this is the practical form of explainability. A confidence percentage or a natural-language justification provided after the decision is not sufficient. The explanation must arise from the very structure of the decision and be understandable to both humans and downstream automation.
CoreSec does not use machine learning in the final decision-making process. ML may be used upstream for anomaly extraction, probe analysis, or log summarization, but the final decision is deterministic. This does not prove that deterministic rules are always better than ML; it shows that, for this specific high-cost action path, the authors prioritized predictability, abstention, and auditability.
Production gate for safe AI abstention
Do not allow irreversible actions without a complete evidence trail
Before an AI workflow issues a refund, restricts an account, changes a budget, or activates a mitigation, define which signal is requisite, which is sufficient, and which discrepancy leads to abstention, which person takes charge, and what audit trail will document the decision.
Seven Steps to Safer Business AI
Cross-network data transfer must be handled with care. The P2.15, two-thirds, and 4% thresholds are not a one-size-fits-all solution for e-commerce, marketing, fraud, or support. The generalizable aspect is the architectural approach: different sources of evidence have different coverage, latency, and authority, so they should not simply be averaged.
Seven Steps from the Use Case to a Controlled AI Workflow
Step 1Here is a decision and the cost of error
Describe the exact action, who it affects, and what the cost is of a false positive, a false negative, or a delay. The abstention policy is based on the failure cost.
Step 2Map the failure surfaces
Distinguish between different scenarios, such as refunds, fraud flags, lead routing, or incident mitigation. Don't expect a single overall score to capture all types of risk.
Step 3Assign a role to each source
Indicate which evidence is requisite, required, sufficient, or optional for each case. Record the owner, source system, freshness window, and known coverage gaps.
Step 4Here is an explicit example of an indeterminate condition
Distinguish between missing, stale, and conflicting evidence. Each type of abstention must have a specific fallback, not a general «the AI failed.».
Step 5Set up escalation and permissions
Specify which user is in charge, what data they can view, and which actions are blocked. In high-risk situations, the default should be “pause” or “reversible draft.”.
Step 6Try delay, conflict, and absence
Don't just test the "happy paths." Use simulations and staged tests for late APIs, missing orders, conflicting records, and stale policies; the Simulation-based testing in MCP and RAG workflows shows why pre-production testing is essential.
Step 7Measure quality and address data gaps
Monitor false positives, false negatives, abstention rate, time-to-resolution, wrong-action cost, and abstention ratio. Use abstention clusters to improve specific connectors and agents.
In marketing attribution, a CRM conversion, a consented analytics event, and a modeled touchpoint should not be treated as equivalent votes. In fraud review, the absence of mandatory verification is not the same as a low risk score. In customer support, AI should not issue a refund or impose a restriction when the order and payment systems disagree. For more examples of thresholds, see AI Workflows in Marketing with Human Oversight show where the final approval remains a human decision.
The KPI isn’t just «how many cases the AI closed automatically.» We need precision per action, the cost of false positives, escalation time, trace quality, and the ability of abstentions to systematically reveal data gaps. If the team reduces abstentions by relaxing the thresholds, it may simply recreate the forced-classification problem.
CoreSec also has clear limitations. New firmware crashes or unknown forms of optical degradation may require new agents. Pipeline failures continue to cause missing data, while large-scale events deliberately lead to mass abstention. The baseline comparison is made with the previous production system and not with all academic alternatives.
The most useful conclusion is not that every company should copy four flags and three thresholds. It is that the reliability of a decision system is not measured by how often it responds, but by whether it knows which evidence is decisive, whether it keeps uncertainty visible, and whether it prevents a weak indication from turning into a precise or irreversible action.
TWO DOTS applies this practical approach to business automation and AI: mapping the actual process, clearly defining inputs, conditions, approvals, error handling, logs, and human checkpoints before a workflow gains greater autonomy.
From the AI demo to safe mode
Design a workflow that knows when to stop
TWO DOTS maps data sources, decision roles, abstention rules, approvals, and audit trails so that AI and business automations can save time without obscuring uncertainty or performing incorrect actions.
It is a production-grade root cause analysis system for large CLOS networks. It synthesizes multiple telemetry sources using PAM-inspired control flags and allows for explicit abstention when the evidence is insufficient.
What does "AI abstention" mean?
This means that the system does not provide a definitive answer or take action when critical data is missing, stale, or conflicting. Any decision to refrain from action must be accompanied by an explanation and a safe escalation process.
Why isn't a confidence score enough?
Because a score can encompass different scenarios, such as a lack of evidence, conflicting sources, and a decisive indication, CoreSec preserves these distinctions within the structure of the decision.
What are the four control flags?
"Requisite" for critical prerequisites, "required" for elements that must be met, "sufficient" for decisive signals, and "optional" for supporting indicators.
By how much has the false positive rate improved?
The paper reports a rate of less than 1% using CoreSec, compared to a historical baseline of 18 to 22 percent for the previous weighted approach, based on a comparison covering 712,345 incidents. This was not a parallel A/B test on the same dataset.
Does CoreSec use machine learning?
Machine learning can be used upstream to extract signals, but the final synthesis is deterministic. This allows for predictable abstention and a verifiable decision trace.
Can the same logic be applied to e-commerce or customer support?
Yes, in principle: role-based evidence, explicit opt-out, and escalation. These specific CoreSec thresholds are derived from Azure Clos fabrics and should not be transferred as-is to another workflow.
What is the first practical step for a business?
Select a decision with a known cost of error, map out the sources of evidence, and determine which missing information or disagreement is blocking the action and who will take charge.