The Minds Behind the Models: Mathurin Aché Arnaud

Mathurin Aché Arnaud: The Engineer Who Built a System to Catch His Own Overfitting

Mathurin Aché Arnaud doesn't come from finance. He comes from fraud detection — the engineering discipline of building machine learning models that must survive contact with adversaries who adapt the moment you deploy. It's a world where the class you care about is rare, the costs of being wrong are asymmetric, and a model that looks convincing in-sample will quietly degrade once it meets reality.

"Every day you're negotiating between a metric that looks good on a validation set and a system that has to hold up against people actively trying to break it. That's excellent preparation for quantitative finance, because it teaches the same core suspicion: a model that looks convincing in-sample will degrade quietly once it meets reality, and your job is to know how much, before it costs you."

What pulled him toward quant specifically was the feedback loop. In most applied data work, you never really find out whether you were right — the counterfactual is unobservable, and everyone agrees the model "helped." A competition with a held-out proprietary test set is brutally honest by comparison.

"AlphaNova was the first setting I'd found where being wrong was cheap, fast, and precisely measurable. For an engineering temperament, that's addictive."

Read Mathurin’s Unfiltered Answers Below

Mathurin’s responses offer a rare, unfiltered look inside the mind of a deeply rigorous data scientist. We’re grateful to him for sharing such detailed reflections - so much so that we’ve included his answers in full, unedited, at the bottom of this post. Un grand merci, Mathurin.

The Self-Improving Search Loop

Mathurin's standout project from Competition 5 wasn't a single model — it was a research system designed to correct itself. It started with a troubling observation: his early signals scored around 0.079–0.081 on his own walk-forward validation, but the server returned 0.036–0.043. Roughly half.

"Most participants treat that haircut as bad luck. I decided to treat it as the actual object of study: if there's a systematic transformation between my validation number and the real one, then optimising the validation number is optimising the wrong thing."

So he built a loop with three components:

A fast evaluator that reproduced the official runner exactly — to the fourth decimal — so a candidate signal could be scored in a fraction of a second instead of minutes.
A quality-diversity search (MAP-Elites) over signal architectures, warm-started from an archive so knowledge accumulated across runs.
A surrogate model of the validation-to-test haircut, refitted every single time a new server result came back.

The search didn't maximise validation Sharpe. It maximised predicted test Sharpe. Every submission stopped being just an attempt and became a labelled observation about the environment.

But the most instructive moment wasn't a success — it was a failure of his own meta-analysis.

The Meta-Overfitting Trap

With three server results in hand, Mathurin's surrogate model had learned a clean, satisfying story: simpler signals — fewer legs, round weights, no fine sweeping — transfer better than heavily optimised ones. It matched quant folklore. It was elegant. He believed it enough to redesign his candidates around it.

Then he accumulated more consistent results and checked properly.

"One of my three points had been measured against an older test window. Once the six coherent results were on the table, the haircut was flat — 0.56 ± 0.02, independent of complexity — and test score correlated +0.94 with validation score. My elegant theory was an artefact of one contaminated observation."

He had built a system specifically to protect him from overfitting, and he overfitted the meta-analysis — three points and a plausible narrative.

"The fix wasn't cleverness, it was refusing to trust a conclusion drawn from three observations no matter how good it sounded, and having the infrastructure to invert the whole search once the data said so."

This is the core lesson of Mathurin's competition: even your own conclusions about your own process need to be tested with the same rigour you apply to your models.

The Fast Evaluator: An Unlikely MVP

When asked what tool or technique he found himself reaching for most, Mathurin didn't name a model. He named a fast evaluator.

"It's the least glamorous artefact I built and by far the highest-leverage one. Once you can score a candidate in a fraction of a second, and you've verified it reproduces the official pipeline exactly, the entire economics of your research change. You stop reasoning from intuition about which idea deserves an expensive run, and start testing systematically. Nearly everything I learned in this competition came downstream of that."

This led to a discovery that killed an entire branch of his work in one line: the pipeline applies a cross-sectional rank before scoring.

"That makes any temporal rescaling of a signal completely inert — a per-row rank is invariant to positive renormalisation. I'd spent real effort on dispersion-scaling schemes that modulated signal intensity over time. All of them were mathematically guaranteed to do nothing. Only the cross-sectional shape matters. You find that kind of thing by reading the evaluation code carefully, not by tuning."

Two Dead Ends That Taught More Than Success

Mathurin described his process honestly, including the parts that failed.

Dead end one: a novelty metric that was fiction. AlphaNova rewards signals that are distinct from the existing population, so he built local tooling to measure that distance and used it to steer candidate selection.

"One signal measured 16° of separation locally and 3.8° on the server. Another measured 37° locally and 7.7° in reality. The local reference set simply didn't reflect the actual field. I spent weeks navigating by an instrument that wasn't connected to anything, and eventually deleted the heuristic entirely. If you can't calibrate a measurement against ground truth, it isn't a measurement."

Dead end two: a rejection that taught him more than any acceptance. He submitted a deliberately orthogonal, low-correlation signal — genuinely causal, no leakage — and it was rejected for overfitting.

"Its information coefficient was simply too close to noise. The lesson generalises well beyond this platform: an overfitting check isn't only a leakage check. A signal statistically indistinguishable from nothing is 'overfit' in the sense that matters, because you've fitted the only structure it has. Don't buy decorrelation with significance."

The Mono-Alpha Discovery

Late in the competition, Mathurin ran an exhaustive alpha survey — six features, momentum and reversal across nine horizons, volatility, differences, spreads — to answer a question he'd been avoiding: how many genuinely distinct sources of return exist in this dataset?

"The answer was approximately one. A single reversal effect dominates everything, and exactly one feature is meaningfully orthogonal to it, with a PnL correlation near zero."

This reframed his entire endgame.

"Near the top of a leaderboard on a mono-alpha dataset scored over a short live window, the remaining spread between participants is mostly noise, not modelling skill. Once you accept that, your job changes: stop grinding for a fourth decimal of validation Sharpe that won't survive, secure the one genuinely orthogonal source of return you've found, and manage variance rather than chase it."

His best signal came from exactly that — a validated core plus a modest overlay on the one uncorrelated feature, for about a 6% improvement.

"That's what the data actually permitted. I'd rather report that clearly than dress up noise as edge."

AI as Engineering Multiplier, Not Idea Generator

Mathurin uses Claude Code heavily, but in a specific role.

"The entire research infrastructure — the exact-reproduction evaluator, the quality-diversity search, the surrogate model, the feedback command that refits everything when a new server result arrives — was built in close collaboration with it. AI didn't find my alpha. It made running hundreds of disciplined experiments cheap enough that I could afford to be systematic, and it kept a written, versioned record of what I'd already disproven."

The failure mode he guards against is that these tools are exceptionally good at producing plausible narratives.

"My 'simplicity predicts transfer' theory was exactly that — a coherent, well-argued story fitted to three data points. It was wrong, and no assistant was going to catch it for me, because it was internally consistent and matched prior expectations. That's precisely the class of error that survives review."

So he settled on a clear division of labour:

"The assistant handles implementation, refactoring, and keeping the experimental record honest. Causality checks, data alignment, and — above all — deciding what counts as sufficient evidence stay firmly on my side of the line. The decision about whether something is real still has to come from the data, and from someone willing to be disappointed by it."

What We Can All Learn

Mathurin's profile is full of transferable lessons. Three stand out:

Build the infrastructure to test cheaply. A fast evaluator changes the economics of your research. When experiments cost seconds instead of minutes, you can afford to be systematic instead of intuitive.
Don't trust your own narratives. Even a sophisticated researcher can overfit a meta-analysis of three points. The cure isn't being smarter — it's refusing to trust conclusions drawn from small samples, no matter how good the story sounds.
Report what the data actually permits. Mathurin's best finding was a negative one: this dataset has roughly one dominant source of return. He chose to report that clearly rather than dress up noise as edge. That intellectual honesty is rarer than it should be.

Stay tuned for more profiles from the AlphaNova community. If you'd like to be featured, reach out—we'd love to share your story.

Join the latest AlphaNova competition and see if your signals can survive the same rigorous tests that Mathurin's did.

Full Interview: Mathurin Aché Arnaud

Q1. How did you first get into quantitative finance or data science?

My path came from the engineering side rather than from finance. At AdvanThink I work on fraud detection for the banking sector — specifically, optimising the machine learning models that decide, in real time, whether a transaction is legitimate. It's an unforgiving problem: the class you care about is rare, the adversary adapts to your model the moment you deploy it, and the costs of a false positive and a false negative are wildly asymmetric and both very real. Every day you're negotiating between a metric that looks good on a validation set and a system that has to hold up against people actively trying to break it.

That's excellent preparation for quantitative finance, because it teaches the same core suspicion: a model that looks convincing in-sample will degrade quietly once it meets reality, and your job is to know how much, before it costs you.

What pulled me specifically toward quant was the feedback loop. In most applied data work you never really find out whether you were right — the counterfactual is unobservable and everyone agrees the model "helped." A competition with a held-out proprietary test set is brutally honest by comparison: you make a claim, the environment answers with a number, and there's nowhere to hide. AlphaNova was the first setting I'd found where being wrong was cheap, fast, and precisely measurable. For an engineering temperament, that's addictive.

Q2. What's a project you've worked on recently that you're excited about?

The one I'd point to is the self-improving search loop I built for Competition 5 — and I'm excited about it less for what it found than for what it forced me to admit.

The starting observation was a gap. My early signals scored around 0.079–0.081 on my own walk-forward validation and came back from the server at 0.036–0.043. Roughly half. Most participants treat that haircut as bad luck. I decided to treat it as the actual object of study: if there's a systematic transformation between my validation number and the real one, then optimising the validation number is optimising the wrong thing.

So I built a loop with three parts. A fast evaluator that reproduces the official runner exactly — to the fourth decimal — so a candidate signal can be scored in a fraction of a second instead of minutes. A quality-diversity search (MAP-Elites) over signal architectures, warm-started from an archive so knowledge accumulates across runs. And a surrogate model of the validation-to-test haircut, refitted every single time a real server result comes back. The search doesn't maximise validation Sharpe; it maximises predicted test Sharpe. Every submission stops being just an attempt and becomes a labelled observation about the environment.

The part I'd actually put on a slide is the moment it caught me being wrong. With three results in hand, the model had learned a clean, satisfying story: simpler signals — fewer legs, round weights, no fine sweeping — transfer better than heavily optimised ones. It's an appealing thesis, it matches quant folklore, and I believed it enough to redesign my candidates around it. Then I accumulated enough consistent results to check it properly, and discovered one of my three points had been measured against an older test window. Once the six coherent results were on the table, the haircut was flat — 0.56 ± 0.02, independent of complexity — and test score correlated +0.94 with validation score. My elegant theory was an artefact of one contaminated observation.

That's the real lesson of the project. I built a system specifically to protect me from overfitting, and I overfitted the meta-analysis — three points and a plausible narrative. The fix wasn't cleverness, it was refusing to trust a conclusion drawn from three observations no matter how good it sounded, and having the infrastructure to invert the whole search once the data said so.

Q3. What's one tool, library, or technique you've found yourself reaching for a lot lately?

Not a model — a fast, exact evaluator.

It's the least glamorous artefact I built and by far the highest-leverage one. Once you can score a candidate in a fraction of a second, and you've verified it reproduces the official pipeline exactly, the entire economics of your research change. You stop reasoning from intuition about which idea deserves an expensive run, and start testing systematically. Nearly everything I learned in this competition came downstream of that.

One technical detail I enjoyed, because it invalidated a whole branch of my work in one line: the pipeline applies a cross-sectional rank before scoring. That makes any temporal rescaling of a signal completely inert — a per-row rank is invariant to positive renormalisation. I'd spent real effort on dispersion-scaling schemes that modulated signal intensity over time. All of them were mathematically guaranteed to do nothing. Only the cross-sectional shape matters. You find that kind of thing by reading the evaluation code carefully, not by tuning.

On the modelling side, the technique I keep returning to is unglamorous too: fixed-weight linear combinations of engines that were each validated independently. For context, the official regularised gradient-boosting baseline reached about 0.014 Sharpe on this data. Hand-specified linear blends reached 0.086. That ratio tells you where the information actually lives.

Q4. In the AlphaNova competition, what strategies or modeling approaches have you been using to generate your signals?

I'll describe the process honestly, including the two dead ends, because they were more instructive than the successes.

The approach. Build a small library of interpretable "engines" — reversal effects, cross-sectional bucket structures, feature interactions — validate each one independently on a walk-forward split, then combine them with fixed weights. Then search over architectures, not against validation Sharpe but against predicted out-of-sample Sharpe, using the haircut model described earlier.

Dead end one: my novelty metric was fiction. AlphaNova rewards signals that are distinct from the existing population, so I built local tooling to measure that distance and used it to steer candidate selection. One signal measured 16° of separation locally and 3.8° on the server. Another measured 37° locally and 7.7° in reality. The local reference set simply didn't reflect the actual field. I spent weeks navigating by an instrument that wasn't connected to anything, and eventually deleted the heuristic entirely. If you can't calibrate a measurement against ground truth, it isn't a measurement.

Dead end two: a rejection that taught me more than any acceptance. I submitted a deliberately orthogonal, low-correlation signal — genuinely causal, no leakage of any kind — and it was rejected for overfitting. Its information coefficient was simply too close to noise. The lesson generalises well beyond this platform: an overfitting check isn't only a leakage check. A signal statistically indistinguishable from nothing is "overfit" in the sense that matters, because you've fitted the only structure it has. Don't buy decorrelation with significance.

The finding I'm proudest of, and it's a negative one. Late in the competition I ran an exhaustive alpha survey — six features, momentum and reversal across nine horizons, volatility, differences, spreads — to answer a question I'd been avoiding: how many genuinely distinct sources of return exist in this dataset? The answer was approximately one. A single reversal effect dominates everything, and exactly one feature is meaningfully orthogonal to it, with a PnL correlation near zero.

That reframed the whole endgame. Near the top of a leaderboard on a mono-alpha dataset scored over a short live window, the remaining spread between participants is mostly noise, not modelling skill. Once you accept that, your job changes: stop grinding for a fourth decimal of validation Sharpe that won't survive, secure the one genuinely orthogonal source of return you've found, and manage variance rather than chase it. My best signal came from exactly that — a validated core plus a modest overlay on the one uncorrelated feature, for about a 6% improvement. That's what the data actually permitted. I'd rather report that clearly than dress up noise as edge.

Q5. How, if at all, are you using AI tools in your workflow these days?

Heavily, and in a fairly specific role.

I use Claude Code as an engineering multiplier, not an idea generator. The entire research infrastructure — the exact-reproduction evaluator, the quality-diversity search, the surrogate model, the feedback command that refits everything when a new server result arrives — was built in close collaboration with it. That's the real unlock. AI didn't find my alpha. It made running hundreds of disciplined experiments cheap enough that I could afford to be systematic, and it kept a written, versioned record of what I'd already disproven, which matters enormously over a multi-week campaign where the main failure mode is re-deriving a conclusion you'd already reached and forgotten.

The failure mode I actively guard against is that these tools are exceptionally good at producing plausible narratives. My "simplicity predicts transfer" theory was exactly that — a coherent, well-argued story fitted to three data points. It was wrong, and no assistant was going to catch it for me, because it was internally consistent and matched prior expectations. That's precisely the class of error that survives review.

So the division of labour I've settled on: the assistant handles implementation, refactoring, and keeping the experimental record honest. Causality checks, data alignment, and — above all — deciding what counts as sufficient evidence stay firmly on my side of the line. The decision about whether something is real still has to come from the data, and from someone willing to be disappointed by it.