SIM-Swap Fraud Risk Engine: A Short MVP for Transaction Risk Review

Fraud systems are rarely hard because one signal is difficult to score. They are hard because the useful signal usually appears across several weak signals that only become meaningful together.

A payment can look normal in isolation. A new device can be legitimate. A SIM-change event can be harmless. A customer can travel, change phones, retry authentication, and still be acting normally. The problem begins when these events cluster around a high-value transaction or a change in account behaviour.

That was the motivation for this short MVP: a SIM-swap fraud risk engine for transaction review. The repository was built as a small hackathon-style proof of concept, not as a production fraud platform. The goal was to make the shape of the system visible: collect events, score risk, explain the score, and give a reviewer enough context to decide what should happen next.

The useful lesson was less about any single technology choice and more about system design. Fraud workflows need explainability, reviewability, and operational discipline. A score without context is not enough.

The Problem

SIM-swap fraud is especially awkward because the risky event often happens outside the payment system. A telco-side change can create an opening for account takeover, but the bank, wallet, or merchant may only see the downstream effects: login attempts, device changes, payment attempts, failed authentication, or suspicious locations.

That means the payment system needs to reason over correlated signals:

• Has a SIM-related event happened recently?
• Is this a new or low-trust device?
• Is the transaction amount unusual for the customer?
• Are there repeated attempts across accounts or devices?
• Is the geography plausible?
• Should the user be blocked, challenged, or routed to review?

The MVP treated the problem as an event interpretation workflow rather than a simple “model predicts fraud” problem.

How I Thought About Solving It

The first design decision was to separate the entities that are easy to blur together:

• Payment transactions: amount, status, merchant, timestamp, and risk state.
• Device fingerprints: trust score, device change history, and activity.
• Risk assessments: the score and the reasons behind it.
• Fraud alerts: the operational unit a security team actually reviews.

Keeping these concepts separate makes the system easier to reason about. It also makes the eventual reviewer experience much stronger because the UI can show why a case was flagged rather than hiding everything behind one opaque number.

The second decision was to keep AI in the right part of the workflow.

For this kind of system, I would not make an LLM the final fraud boundary. The LLM is useful as an investigation assistant: it can summarise alert context, highlight suspicious patterns, and suggest immediate, short-term, and long-term actions. But blocking payments, changing authentication state, or escalating a customer should still be governed by explicit rules, policy, thresholds, and human review.

In the MVP, the AI layer was therefore positioned around investigation and explanation rather than final decision-making.

What The MVP Covered

The project used a full-stack Next.js application with a consolidated dashboard. The dashboard grouped the workflow into four areas:

• Overview: security score, active alerts, recent activity, and agent status.
• Transactions: payment history, filtering, and risk review.
• Security monitoring: alert context, device trust, and risk analytics.
• Payments: a protected payment flow that could generate events for the engine.

The backend model included enough structure to support the core workflow:

• transaction monitoring,
• SIM-swap-style alert signals,
• device-trust scoring,
• risk assessment records,
• AI-assisted investigation summaries,
• and a review path for high-risk cases.

The point was not to make every feature deep. It was to show an end-to-end slice: a transaction enters the system, signals are interpreted, a risk posture is produced, and a reviewer sees enough context to act.

Why The Dashboard Matters

Fraud tooling is not only a modelling problem. It is an operator problem.

If a reviewer has to move between too many pages, reconstruct context manually, or read raw logs to understand why a case was flagged, the system will either be ignored or will create unnecessary friction. That is why the MVP consolidated transactions, alerts, payment events, and risk monitoring into one dashboard with tabbed sections.

That choice sounds like a UI detail, but it is really an operational choice. A fraud workflow needs to reduce the time between signal and action. The interface should help the reviewer answer:

• What happened?
• Why is it suspicious?
• What evidence supports the alert?
• What should be done now?
• How confident is the system?

Those questions shaped the dashboard more than the visual design did.

Backtesting The Risk Engine

The right way to validate this kind of system is not to look at a few demo transactions and decide that the output feels plausible. It needs a backtest.

The backtest I would run is event-time based:

1. Replay historical payment events in timestamp order.
2. Join each transaction with only the SIM, device, authentication, and location events that would have been known at that moment.
3. Score the transaction using the MVP rules and thresholds.
4. Compare the score against confirmed fraud labels and known legitimate transactions.
5. Measure precision, recall, false-positive burden, time-to-detection, and estimated value at risk.
6. Review the highest-risk false positives to understand where the system is too aggressive.

The most important constraint is avoiding look-ahead leakage. A fraud engine cannot use future information to decide whether a payment should be challenged now. That is why the replay must respect event time, not just row order in a dataset.

I would also separate model quality from operational quality. A rule can have good recall and still be unusable if it creates too many manual reviews. In fraud, review capacity is part of the system.

POC Boundary

This was a short-term POC/MVP. It was not connected to live telco data, production banking rails, regulatory workflows, confirmed fraud labels, or an analyst operations team.

A production version would need:

• real labelled fraud and non-fraud data,
• privacy and regulatory review,
• telco and authentication event contracts,
• strict audit logging,
• human-review tooling,
• threshold governance,
• false-positive monitoring,
• and incident response runbooks.

Without those pieces, the responsible claim is that this was an architecture and workflow exploration, not a deployable fraud product.

What I Learned

The main learning was that explainability is not an optional layer in fraud systems. It is part of the product.

If the system says “high risk” but cannot explain which signals contributed, the reviewer cannot trust it. If the system explains the signals but gives no operational recommendation, the reviewer still has to do too much work. If the system recommends action but hides uncertainty, it can become dangerous.

The better pattern is a reviewable workflow: structured signals, explicit rules, context-aware summaries, measured outcomes, and a human path for consequential decisions.

That is the part of the MVP I would keep.

Repository: github.com/ThamuMnyulwa/bet-hackathon-2025