Coinbase announced that an artificial intelligence agent helped it slash fraudulent payments by 30% in a controlled experiment over Onramp its service for buying crypto from third-party apps. It also saw a 22% decrease in the value of fraudulent transactions. The figure was achieved after Coinbase added an AI risk agent to its existing machine learning and rule-based systems.
Coinbase’s CEO Brian Armstrong wrote about the experiment on October 12, describing how the company fine-tuned Qwen3.5-9B on its data and concluded that a model optimized for a specific task may perform better than a large language model (LLM) with a broad range of functions. According to Armstrong’s post, Coinbase used an open-weight model that excelled at the customized task at a lower computational cost and latency than alternative closed-source models.
We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case).
I think we’ll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you’ll see specialist models out perform on more narrow tasks, either working on teams, or by themselves.
We’re making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them:
“/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started”
I imagine we’ll have some backend services that could be replaced entirely by this new architecture.
— Brian Armstrong (@brian_armstrong) October 11, 2026
Onramp is a Coinbase product that enables users to purchase crypto via “guest checkout” in third-party apps that utilize the service. During the experiment, Coinbase’s AI agent analyzed transactions and their sequences to determine whether they are fraudulent or not. This allowed it to identify patterns that could be misleading when examining an individual payment.
As a result, the company reported fewer fraudulent transactions at a 30% decrease and a 22% lower value of fraudulent transactions in the test group.
In the experiment, the agent did not act autonomously but instead provided risk scores for transactions, with the app’s code implementing specific rules. It could cancel a transaction if it had passed all previous checks, although it could not undo a canceled transaction.
Coinbase also performed a comparison test to see how different iterations of closed-source models would perform using one standard set of 16,140 transactions from 7,293 users and 813 fraudulent transactions. It found that newer iterations of Opus, Sonnet, and GPT had poorer performance in several metrics compared to older models on the same task.
According to the experiment, GPT-5.6 had a higher precision — the ratio of true positive transactions identified as fraudulent but a lower recall — the ability to identify all fraudulent transactions.
The results prompted Coinbase to adopt an open-weight model as an alternative to proprietary large language models. Unlike an API, which only provides access to another company’s software, an open-weight model allows developers to modify the parameters of the code for their purposes directly.
In its experiment, Coinbase did not find universally better performance from smaller and open-source models. At the same time, the company asserted that a model optimized for a particular task could deliver better results than a general-purpose LLM for the same task.
CertiK conducted its research into the security implications of autonomous agents and concluded that current AI systems “must be designed with safety by default.” It recommended limiting the autonomy of AI agents and establishing the accountability of human operators while suggesting the adoption of such measures as a mandatory standard.
The recommendation follows after CertiK discovered that security researchers have exploited such agents’ autonomy to gain unauthorized access to systems.
Coinbase’s experiment appears to confirm this risk, as its AI system did not take direct action but instead supported human operators by supplying them with risk scores. By comparison, the company’s design philosophy prevents such issues as the described AI agent from being able to cancel a transaction — only the application itself can do that.
At the same time, the experiment only proves that training a model to execute a specific function can yield better results in some circumstances. However, it does not prove that the same result would be achieved in all applications, including a broader use-case within the payment industry.
Stay informed with the latest trends in Web3, blockchain innovation, and cybersecurity updates at 3verseTV
You need to login in order to Like
Leave a comment