← statichum.studio

Predictive Test Selection for Normal-Sized CI Pipelines Without Meta's Infrastructure

dev tool real project •• multiple requests

Meta published Predictive Test Selection in 2018: train a model on historical test outcomes, select the ~30% of tests relevant to a given diff, catch 99.9% of regressions. Seven years later, no off-the-shelf tool brings this to teams outside FAANG. TestImpact.io shut down, Launchable pivoted, Buildkite Test Engine exists but is narrow and expensive, Gradle Enterprise is JVM-only. AI-assisted development is pushing CI bills up 3–5x (more PRs, more agents, more commits) and a December 2025 Ask HN thread explicitly asks for 'an LLM tool that can sit on a CI pipeline to propose what tests should be blocking.'

builder note

Forget the LLM framing — the original Meta approach is a gradient-boosted decision tree, which is fine. What's new is 'GitHub Actions reusable workflow you add in 3 lines, we slurp your coverage data + PR history, we send back a set of test IDs to run.' Monetize per-CI-minute saved; that pricing sells itself to the CFO.

landscape (5 existing solutions)

The technique is seven years old and openly published. Nobody has turned it into a product a 15-engineer team on GitHub Actions can drop in with an action reference. The CI-bill-shock from AI-generated PR volume is forcing this conversation right now — every team with a 40-minute test suite is quietly bleeding.

Buildkite Test Engine Works well, but locked to Buildkite pipelines. Teams on GitHub Actions / CircleCI / GitLab have no equivalent.
Launchable (pivoted) Was the most promising independent player. Pivoted toward enterprise DevOps consulting, effectively leaving SMB / OSS unserved.
Gradle Develocity (formerly Gradle Enterprise) Predictive test selection exists for JVM/Maven/Gradle. Polyglot or Python/Rust/Go teams don't get it.
Nx affected Purely graph-based: runs tests for projects whose code changed. Doesn't do the ML 'this test has historically caught bugs in this path' step.
Bazel rules_test + test sharding Can skip unaffected targets via the build graph, but requires full Bazel migration — a cost nobody pays just for test selection.

sources (4)

other https://engineering.fb.com/2018/11/21/developer-tools/predic... "catches more than 99.9 percent of all regressions... running just a third of all tests" 2018-11-21
hn https://news.ycombinator.com/item?id=46345827 "LLM tool that can sit on a CI pipeline to propose what tests should be blocking" 2025-12-27
other https://repositum.tuwien.at/handle/20.500.12708/215633 "Predictive test selection: a replication study" 2024-06-15
other https://buildkite.com/platform/test-engine "Test Engine selects only the tests affected by a given change" 2026-02-01
ci-cdtestingpredictive-test-selectiongithub-actionsregression-testing