How Crucible works
Your strategy fired on hundreds of trades. Some won, some lost. Crucible looks for what the winners had in common, gives every trade a score out of 100, and then checks whether skipping the low-scoring ones would actually have helped.
The check is the part that matters, and it is the part most tools skip. Everything below is how it is done.
Five steps, in order, with real numbers at every one. The strategy drawn here is the one the landing page demo opens on: buy SPY when the open gaps up more than 0.5% from the previous close, exit when the close clears its 5-day average or after three bars.
That rule fired 939 times between 1993 and 2026. Every figure below is measured off those trades, and the prices and the generator are both in this repo, so none of it has to be taken on trust.
You bring the log
One row per trade the rule fired. Winners and losers, in the order they happened. Each row carries what you could see at the entry bar, then what happened next.
| What you could see at the entry bar | What happened next | ||||
|---|---|---|---|---|---|
| RSI(14) | From 200-day avg | Gap | Down days | Result | Return |
| 50.3 | +2.37% | +0.54% | 0 | Win | +0.067% |
| 35.7 | −3.12% | +1.00% | 0 | Loss | −0.139% |
| 42.6 | −3.77% | +0.63% | 0 | Loss | −0.280% |
| 63.9 | −0.52% | +0.69% | 0 | Win | +0.408% |
| 47.5 | +0.64% | +0.54% | 0 | Loss | −0.504% |
| 33.2 | −1.70% | +0.51% | 4 | Win | +0.345% |
| and 933 more, every one the rule fired between 1993 and 2026 | |||||
Both percentages are distances from a level, not prices: the close against its own 200-day average, the open against the previous close. Each is a percentage of the level it is measured from.
Scroll the table sideways for how each row ended.
It gets cut in two
Sorted oldest to newest, then split. The older 70% is what the model may learn from. The newer 30% is sealed, and not opened until there is a Trade Quality Rating and a threshold to test on it.
939 trades, oldest first
cut at the 70% mark, by date and never at randomEvery weight the model learns, and the line it is eventually cut at, come from these trades and nothing else.
Opened once at the very end, to test the answer. Nothing here shapes the model or moves the line.
The model weighs the columns
Each factor gets one weight: how far it leaned toward winners in the in-sample trades, and which way. That is the whole model.
What the model learned from the 657 in-sample trades
- consecutive down daysmore is better
- distance from the 50-dayfurther below is better
- distance from the 20-day highcloser to the high is better
- medium-term momentum (RSI 14)lower is better
- the size of the gapsmaller is better
and five more, all smaller. Distance from the 200-day is one of them: it is in the file, and the model gave it almost nothing.
Every trade gets a score
Multiply each factor by its weight, add them up, squash the total onto 0 to 100. Arithmetic you could do by hand, done 939 times.
Scoring row 6 of the table in step 1
- 4 down days, well above this file’s average
- +0.44
- RSI 33.2, well below average, and lower is better here
- +0.11
- the other eight factors, together
- +0.01
- where every trade starts, before any factor
- +0.47
- total
- +1.03
And the same for the other 938
Every trade in this file scores between 47 and 81.
Then the line is tested
Every cut from 1 to 99 was tried on the in-sample trades, and 57 scored best. Here is that line on the 282 the model was never allowed to see.
Out-of-sample, never seen by the model. 282 trades.
22 dropped, 260 keptWhy your log gets cut in two
Sort every trade by date. The oldest 70% is the part the model may learn from. The newest 30% is put away and not looked at again until there is an answer to test.
The reason is that finding a rule which fits trades you have already seen is easy, and it is easy whether or not the rule means anything. Give any model enough columns and it will describe the past beautifully. The only interesting question is what that rule does next, and the sealed-off newer trades are the closest thing to next that a historical file contains.
So the split is by time, never at random. A random split would let the model learn from trades that happened after the ones it is tested on, which is not a mistake you could make in real trading and not one worth simulating. Your report names both windows in real months, so you can see which of your own trades fell on each side.
If your export is newest-first, that is handled: the dates are sorted before anything is split. If your file has no date column at all, the row order is used instead, and the report says so, because that is a weaker guarantee and you should know when you are relying on it.
What the Trade Quality Rating actually is
It is a scorecard. Each of your factors gets one weight, worked out from the in-sample trades: how strongly that column leaned toward winners, and in which direction. Higher RSI might earn a small negative weight, being further above the 200-day a small positive one.
To score a trade, take its factor values, multiply each by its weight, add them up, and squash the total onto a 0 to 100 scale. That is the whole calculation, and the report prints the weights and the arithmetic so you can reproduce any score by hand.
The number ranks your trades against each other. It is not a probability. A 72 does not mean a 72% chance of winning, and two trades rated 60 and 70 are not eight points apart in any meaningful unit. The fit deliberately pulls every score toward your own average, which keeps the ordering honest and makes the level meaningless. Read the order, ignore the number.
Sorted by rating, your log has the trades the model liked most at the top. A threshold is just a line drawn through that list, and everything below the line is what a filter would have skipped.
The method is logistic regression, which is a standard supervised model and the reason the output lands between 0 and 100. Knowing that changes nothing about how to read the score, which is why it is the last sentence here rather than the first.
What the model cannot see
Each factor gets one weight pushing in one direction. That is a real constraint and it is worth understanding, because it is the difference between no edge found and no edge.
It can learn “lower RSI tends to be better”. It cannot learn “low RSI works, but only when volatility is high”, because that is a combination and the model adds its factors up rather than crossing them. It also cannot learn “this factor pays at both extremes and not in the middle”, because one weight can only point one way.
Both of those are real trading patterns, and both are invisible here. So when Crucible says no edge, the honest reading is “nothing found in the shape this model can hold”. It never means there is nothing there.
There is a way round it, and it is yours to apply. If you believe a factor works at both ends of its range, give the model the distance from the middle instead of the raw value. If you believe two conditions matter together, multiply them into a column of their own before you upload. The model will read either one happily. It just cannot invent them for you.
The threshold, by contrast, is a genuine search: every cut from 1 to 99 is tried on the in-sample half and the best risk-adjusted one is kept. Searching biases a number upward, which is exactly why the result is then reported against the half that took no part in the search.
How many factors is too many
More columns is not free. A model spends evidence estimating each weight, and what it actually spends is instances of the rarer outcome. Two hundred wins do not help a model asked to learn from nine losses.
So the number that matters is the rarer outcome divided by the number of columns. A log with 205 wins and 156 losses running 25 columns is at about six per factor. The same log run on eight columns is at nearly twenty.
Crucible refuses below 3 per factor, which is a floor and not a recommendation. The long-standing rule of thumb for this kind of model is 10 per factor, and between the two the weights are thin: usable, tested out-of-sample like everything else, and estimated on less than you would want. Your report shows the figure for your own log in the columns panel, so you can watch it move as you switch columns off.
Practically, five to eight well-chosen factors on a few hundred trades beats twenty on the same file. If two of your columns measure nearly the same thing, the model will tell you: it flags when one factor is riding on a stronger one.
The one thing the tool cannot check
Everything here assumes your log is complete: every trade your rules would have flagged over the period, wins and losses alike, with none quietly left out.
A cherry-picked log produces a confident, entirely wrong answer, and there is no statistical repair for it. Crucible watches for the one tell it can see, an implausibly high win rate, and a carefully incomplete log walks straight past that. This is the only input problem the tool hands back to you.
The same applies to your exit. Outcomes have to come from one mechanical rule applied identically to every trade, whether that is a fixed target and stop or a trailing stop. A managed, discretionary exit produces labels the model reads as facts about your entries when they are really facts about your judgement on the day.
What it refuses to claim
It will come back with no edge found
That is the whole point. A tool that always finds something is a tool that finds noise.
Expectancy, not win rate
With uneven payoffs a filter can raise your win rate while what you make per trade falls. Expectancy leads here, with total return beside it so the trades the filter cost you are visible too.
It refuses thin samples
Forty trades, and at least fifteen wins and fifteen losses among them, and more of each as you add factors. Short of either you get “not enough data”, not a confident-looking number that falls apart live.
It keeps leaking columns out of the model
A P&L or exit column records how the trade ended, so as a factor it would let the model read the answer off its own input. Crucible spots those and holds them back. Your P&L is still used, for expectancy.
And what it still cannot tell you
What any single trade will do
The Trade Quality Rating puts your trades in order, best to worst. Read the ordering and ignore the number itself, for the reasons set out above.
What it costs you to trade
Every figure assumes a flat 0.02% round-turn cost, spread, slippage and commission combined, when your log's return is in percent. It does not yet do this for an R-multiple or unlabelled column. Nothing here accounts for position size, and drawdown assumes the same stake on every trade.
The three that matter most before you upload anything are on the front page, and every result carries the ones measured on your own log beside the figures they qualify.
The words on the report
- Trade
- One historical instance where your rules fired, with the conditions at entry and how it finished under your exit. Not a trade you chose to take: every instance, including the ones you would have talked yourself out of.
- Factor
- One column of conditions at the entry bar. RSI at entry, distance from the 200-day, the hour, the volatility. Anything you could have known at the moment the rule fired.
- In-sample
- The older 70% of your trades, which the model is allowed to learn from. Anything measured here is the model describing what it was shown, so nothing measured here is evidence.
- Out-of-sample
- The newer 30%, sealed off. The model never sees these while it learns, and the threshold is chosen without them. They exist to check the answer once.
- Trade Quality Rating
- The 0 to 100 score each trade gets from its own entry conditions. It ranks your trades against each other. It is not a probability, and a 72 is not a 72% chance of winning.
- Threshold
- The line through the ranking. Trades rated above it are the ones a filter would keep. Chosen on the in-sample half only, then tested once on the out-of-sample half.
- Expectancy
- What one trade makes on average, wins and losses together. The headline metric here, because a filter can raise your win rate while lowering this, and the win rate is the one that feels better.
- Verdict
- One of four: edge, conditional edge, no edge found, or not enough data to say. Conditional edge means the filtered trades made money and the unfiltered strategy did not, so the filter is carrying everything.
Now run it on your own trades
No account, and your trade log never leaves the browser.