I built a trading system, tested four ideas for an edge, and found none.

The short version

• What I built. A swing-trading system where the agent may only propose. Deterministic code verifies, sizes and risk-checks every idea, and an append-only ledger records each step. It placed its first real orders in September 2026, one share at a time.
• What I tested. Four ideas for an edge: a Claude agent, classic quant signals, insider buying from SEC filings, and the plain momentum rule the system runs now.
• What I found. No edge. Claude did not beat the free rule and cost about $27 a year, so I removed it. No classic signal survived out-of-sample testing and a correction for luck, and insider buying added nothing once sizing and stops applied.
• The uncomfortable part. The momentum rule returned +66.8% against SPY's +70.1% from 2022 to 2026, at about half the drawdown. Across the classic quant signals I tested, the best deflated Sharpe was 0.65, below my bar of about 0.95. I report that as the result, and the system stays in shadow mode.

The problem

I wanted a system that could run unattended without my having to trust a language model with money, and I wanted to know whether it had an edge at all. Models are fluent and sometimes wrong, so the design question was not "how smart can the agent be" but "how bad can a failure get, and how fast can I see it."

How it is built

In the version I tested, the only step that used a model was trade proposal. Deterministic code verified, sized and risk-checked every proposal. I have since removed the model, and the system now runs a rule-based momentum agent behind the same checks.

Blue marks the only step that was ever a model. After the experiment below, even that step is a rule.

Four decisions that matter

1. The proposer cannot size or send an order

   An independent verifier recomputes what each proposal implies from my own data. It rejects anything ungrounded (every cited evidence ID must exist in this cycle, and the symbol's own price evidence must be cited), stale, priced outside a collar around the quote, with a stop too close or too far in ATRs, without enough reward for the risk, or without edge after costs.

   Position size comes from quarter-Kelly on the stated edge, capped at 2% risk per trade and 6% of equity at risk across all positions. I used a quarter because the agent's confidence figures are not calibrated.

   verifier.py · portfolio.py · risk.py

2. The risk limits are a file the agent cannot write

   Limits live in config/risk_limits.yaml and change only through a human git commit. The file's hash is recorded with every risk decision, so I can tell which limits applied to any trade. Examples: a 5% drawdown breaker, a 2% daily-loss check that blocks new entries, and at most five new trades a day.

3. Execution is idempotent and never retries blindly

   Client order IDs are derived deterministically from the decision, so a repeated submit finds the existing order. Each order is previewed first, and a preview cost that differs from the intended cost is rejected. If the broker's answer is ambiguous, the order moves to a reconcile state and is resolved by asking the broker, not by sending it again. After a fill, a broker-side stop is placed and re-armed if it goes missing.

   execution.py

4. The audit trail is tamper-evident, and the kill switch is a file

   The events and evidence tables are insert-only, enforced by SQLite triggers that abort any update or delete. Each event stores the previous hash and its own, so an edit breaks the chain. Tables such as orders and trades are projections that can be rebuilt from the events plus broker history.

   The kill switch needs no access to the running process: create state/KILL and new orders stop, and open entry orders are cancelled on the next pass. Real orders also need three separate switches to be on, so any one being off turns entries into rejections.

   ledger.py · killswitch.py

Four tests for an edge

Each idea ran on liquid US stocks and ETFs, and each had to beat simply holding the index. None did.

How I kept a negative result trustworthy

• No lookahead, and a test for it. The test rewrites the future of a price series and checks that no past signal moves.
• Walk-forward selection. Settings are chosen on two years of data, scored only on the next six months, then rolled forward.
• A correction for luck. The deflated Sharpe ratio counts every variant I tried, so each new grid raises the bar for the next.
• Stated biases. The universe is today's survivors, which flatters every absolute return, and each report lists that under its caveats.

The LLM study: did Claude earn its place?

I replayed 2024 (about 252 trading days) over the 100 most liquid US stocks plus an S&P 500 ETF, starting from $1,500. Both agents ran through the same verifier, sizing, stops, slippage model and kill switch. Before the run I set the bar: the Claude agent had to beat the baseline on both Sharpe ratio and win rate.

Verdict: it failed my rule. Claude's Sharpe was 1.39 against 1.36 for the baseline, a lead of 0.03 that is noise. SPY's was 1.89. Claude's win rate was also 3.5 points lower. Its one real advantage was roughly half the drawdown. A measured research cycle cost about $0.11, which is about $27 a year, or 1.8% of a $1,500 account, before any benefit.

So I removed the LLM layer, kept the baseline, and kept the harness that told me the truth.

What I would not claim from this

• It is one year, and 2024 was a strong bull market for the benchmark.
• It is one run of a nondeterministic model, so a rerun would give different numbers. The Sharpe gap of 0.03 sits well inside that noise.
• With a $1,500 account, whole-share rounding and fixed order costs affect both agents.
• An early 49-call partial run showed a 65% win rate for Claude on a handful of trades. I discarded it as statistically meaningless once the full year existed.

What went wrong along the way

1. My first comparison was invalid. The harness never loaded the API key, so every model call failed. The backtester swallowed the error as "no trade" and printed a fake 0% win rate for Claude. Now the harness loads its configuration, records the first error of each kind, and aborts if a model agent only produces errors.
2. I killed healthy runs. I was watching the wrong cache directory, so a run waiting on the network looked hung. I stopped it after 40 minutes and stranded about $7.50 of work.
3. The cache was fragile. Its key included account state that depended on earlier model decisions, so only an identical chain of decisions could reuse it.
4. I had no visibility into spend. About $13 looked like nothing until I added a spend ledger with a hard cap, an abort after three failures in a row, and a request timeout.

Where it stands

The system placed its first real orders on 23 September 2026: one share at a time, each with a broker-side stop. I switched live trading off on 1 October. It now runs in shadow mode, which records decisions without sending orders, and the research since says to keep it that way. I would rather say that plainly than tune until a chart looks good.

A paper account seeded from the real one is compared with SPY each week. If I want an edge, the next place to look is a universe where small capital can compete, such as liquid small caps, with a point-in-time list of constituents to remove survivorship bias. Putting a model back in is not the plan.

Read the code

Source on GitHub ↗ (opens in a new tab) LLM backtest report ↗ (opens in a new tab) Quant research report ↗ (opens in a new tab)

An engineering project, not investment advice. Backtest results do not predict live performance.