# I Replaced a $100K Security Audit with a CI Pipeline — And It Caught More Bugs

[Sascha]

When I built UltrafastSecp256k1 — a high-performance secp256k1 cryptography library targeting CPU, CUDA, OpenCL, Metal, ESP32, and a dozen other platforms — I faced a decision every serious crypto library author eventually faces.

"You need a third-party audit."

The quotes I got: $80K–$120K. Two weeks of engagement. A PDF. No contractual accountability for what happens after the next commit.

I couldn't afford it. And honestly, once I understood what I was actually buying, I didn't want it.

So I built something else.

---

The Problem With Snapshot Audits​

The dominant model is:

code → audit firm reviews for 2 weeks → PDF published → trust badge acquired

The structural flaw: it's a snapshot, not a system.

A PDF tells you the state of the code at the moment of review. It says nothing about what happens after the next commit, after a new platform port, after a new protocol feature lands. The "audit passed" badge persists even if the code is completely rewritten.

Consider: Heartbleed lived in OpenSSL for two years. OpenSSL had been reviewed by expert eyes and was trusted everywhere. The problem wasn't that too few people looked — it's that no system continuously checked the specific property that failed.

A missing bounds check. Two years. Production everywhere.

---

What I Built Instead: CAAS​

Continuous Adversarial Audit System.

The core principle: every security claim must be backed by an executable test that runs on every commit.

Not "we believe this is constant-time." But: "ct-verif LLVM pass, Valgrind taint analysis, and dudect statistical timing all pass for this function — on every commit, across x86-64 and ARM64."

Here's what CAAS looks like in practice for UltrafastSecp256k1:

The Numbers​

Metric	Value
Assertions per build	~1,000,000+
Exploit PoC tests	187 files, 171 registered modules
CI workflows	36 GitHub Actions
Nightly differential checks	1,300,000+ vs libsecp256k1 reference
CT verification pipelines	3 independent (LLVM ct-verif + Valgrind + dudect)
Formal proofs	Z3 SMT (17 proofs) + Lean 4 (19 theorems) on SafeGCD
Build matrix	595 combinations (7 arch × 17 config × 5 OS)

The Bug Capsule System​

When a bug is found — by me, by a contributor, by fuzzing, by a new ePrint paper — it becomes a permanent regression test:

{
  "id": "BUG-2026-0001",
  "category": "CT",
  "severity": "critical",
  "title": "CT branch leak in ecdsa_sign_recoverable low-s normalization",
  "fix_commit": "0a93ff4b",
  "affected_functions": ["ct::ecdsa_sign_recoverable"],
  "expected": { "result": "no_timing_leak", "timing_threshold": 10.0 },
  "exploit_poc": true
}

Run python3 scripts/bug_capsule_gen.py capsule.json and it generates:

• A deterministic regression test (.cpp)
• An exploit PoC test (if exploit_poc: true)
• A CMakeLists.txt CTest fragment

The bug can never silently return. The knowledge is encoded in the test suite, not in anyone's memory.

A Real Example: RISC-V CT Leak​

When I ported to RISC-V, the CT verification pipeline caught a timing side-channel on the first run. GCC was optimizing constant-time code into secret-dependent branches — because the RISC-V backend had different optimization behavior than x86-64.

A snapshot audit done on x86-64 would never have seen this. The PDF would say "constant-time: verified." The RISC-V port would ship with a private key leak.

CAAS caught it in the same commit as the port. Bug capsule created. CI gate added. Can never regress.

The Custom Static Analyzer​

dev_bug_scanner.py is a 700-line domain-specific static analyzer with 28 rule classes — including rules that generic tools like Clang-Tidy and CodeQL simply cannot catch:

# CT_VIOLATION: fast:: call in CT-required signing path
# TAGGED_HASH_BYPASS: plain sha256() where BIP-340 tagged_hash required  
# SECRET_UNERASED: Scalar without secure_erase on signing path exit
# RANDOM_IN_SIGNING: getrandom() in RFC 6979 deterministic path

These patterns are invisible to generic analyzers because they require secp256k1 domain knowledge. fast::scalar_mul() in a signing function is valid C++ — but it's a side-channel vulnerability.

Full Reproducibility​

Everything runs locally in Docker:

# Exact GitHub CI environment, locally
docker compose -f docker-compose.ci.yml run --rm pre-push   # ~5 min gate
docker compose -f docker-compose.ci.yml run --rm gh-parity  # Full GitHub parity

# Auditor challenge environment — self-contained, no setup
docker build -f Dockerfile.auditor -t ufsecp-auditor .
docker run --rm ufsecp-auditor  # runs full audit suite

# Bit-for-bit reproducible build verification
docker build -f Dockerfile.reproducible -t uf-repro-check .
docker run --rm uf-repro-check  # compares two independent builds

An external auditor doesn't need to install a toolchain. The Docker image includes libsecp256k1, @noble/secp256k1, coincurve, and python-ecdsa for differential testing — all hash-pinned.

---

What Happened With Real-World Adoption​

Craig Raw, author of Sparrow Wallet, integrated the library into Frigate — a DuckDB-based Silent Payments scanner used for real Bitcoin mainnet transactions.

Real-world scan result: 2× RTX 5090 scans 2 years of Bitcoin mainnet transactions (133M tweaks) in 3.2 seconds using this library. That's ~41.5 million BIP-352 operations per second.

---

What CAAS Does Not Claim​

Full transparency matters:

• No third-party audit yet. This is acknowledged openly. CAAS is designed to make one as efficient as possible when it happens — but it hasn't happened yet.
• GPU CT is code-discipline only. Vendor JIT compilers (CUDA PTX assembler, Metal, OpenCL runtime) transform kernels at runtime. The 3-pipeline formal CT verification applies to CPU only. Production signing always routes through CPU.
• Novel attacks. By definition, no prior PoC covers unknown unknowns. 11 fuzz harnesses and CT analysis are the mitigations.

---

The Actual Cost Comparison​

Approach	Cost	Coverage	Ages?	Accountability?
Snapshot audit	$80–120K	Bounded time window	Yes, immediately	"Reasonable effort"
CAAS + bug bounty	$5–10K bounty pool	Continuous, adversarial	No	CI fails = hard stop

The $100K buys a PDF with "reasonable effort" liability language. $10K in bug bounties buys adversarial researchers with economic incentive to find what breaks — and every finding becomes a permanent CI gate.

---

Fork It​

The entire infrastructure is MIT-licensed and ships with the library:

git clone https://github.com/shrec/UltrafastSecp256k1.git

You get: 171 exploit PoC tests, 36 CI workflows, Docker environment, dev_bug_scanner.py, bug capsule system, source graph, AI memory — the accumulated security knowledge of every platform port, every bug found, every ePrint paper evaluated.

A startup that forks this doesn't start from zero security. They start from a system that has already caught a RISC-V CT leak, a Metal field arithmetic truncation affecting 0.05% of inputs, an OpenCL carry propagation bug, a CT branch leak in ECDSA signing.

That's the compound effect that a PDF can never provide.

---

UltrafastSecp256k1 is open source under MIT. Full documentation at github.com/shrec/UltrafastSecp256k1. Discussions on Discord.

Источник: # I Replaced a $100K Security Audit with a CI Pipeline — And It Caught More Bugs https://dev.to/vano_chkheidze_5071c537e2/-i-replaced-a-100k-security-audit-with-a-ci-pipeline-and-it-caught-more-bugs-1h11