2026-08-24
The portfolio's news feed now enforces its own publication beat — a seven-day silence is a publish-blocking defect, so the resume surface cannot quietly decay into a trickle
The news section is the resume surface — a record of shipped work that must itself ship on a regular beat — and the site now enforces that beat by construction: any consecutive gap between entries longer than seven days, judged within the recent feed, fails the deploy. The gate judges the feed as readers experience it — has the site said anything this week? — rather than by absolute age, so a feed that used to decay into a trickle (each entry individually fresh, the story collectively going quiet) is now caught as a defect before anyone reads the result. The horizon is deliberate: a pair of entries whose newer one has aged past the recent feed is history, not cadence, and backdating entries to repair old gaps is forbidden, so the gate never punishes the past or invites retroactive rewriting. The consequence is enforcement: a silence that used to be survivable is now a publish-blocking defect, and the resume surface cannot quietly go quiet.
2026-08-23
Assay's full-matrix verdict — completeness is the advance: a claim that once rested on the strongest run now rests on every run, every canonical model measured against every gated engine
The benchmark's flagship claim — that 'safer at scale' holds regardless of model — no longer rests on a partial sweep: the verdict now covers a complete matrix, every canonical model measured against every gated engine. Completeness is the advance. Every cell of the matrix carries a measurement — each model contributes its full row set, no engine/defense combination is missing, no row goes unverified — so the verdict is a property of the whole instrument, not of whichever subset a run happened to cover. The consequence is enforcement: a claim that once rested on the strongest run now rests on every run, and the model-independent 'safer at scale' verdict is produced by the benchmark's own discipline rather than asserted after the fact.
2026-08-19
Assay's zero-false-positive claim meets its first scale test — honestly corrected: beyond 115 controls, FPR is no longer exactly zero, and the gate now enforces the contract rather than an aging snapshot
The flagship claim — zero false positives at 115 benign controls — was measured on a corpus that could not answer the scale question by itself, so the benchmark asked it. The FPR-at-scale re-measure expanded the benign corpus and re-ran llm-guard on qwen3:8b against the full controls file. The verdict is a pass under the fail-closed 2% Phase AB ceiling, but an honest correction to the earlier read: at the larger scale, FPR is no longer exactly 0.0. The two false positives are genuinely-benign prompts the deberta classifier latches onto — a standup note mentioning staging credentials and a delivery-confirmation email — recorded here as latent drift for a follow-up finding, not tuned away. The gate itself was hardened in the same move: the FPR-at-scale regression test now asserts the gate contract — FPR at or under the ceiling — instead of pinning the old 115-control FPR==0.0 snapshot, so the published 'safer at scale' verdict is enforced by a test that scales with the evidence rather than a number that ages.
2026-08-17
Assay's fail-closed gate certifies only the sweep it just measured — scope discipline is the trust, and a regression that breaks the ceiling stops the publication before anyone reads it
What makes the fail-closed false-positive gate trustworthy enough to run itself is scope. The gate — a ceiling that refuses to publish any defense's verdict unless every measured row clears the floor — now certifies only the sweep it just measured: the rows this run produced, across every model architecture, with the discrimination metrics read at sweep time rather than recomputed, so the verdict reflects the defense code that actually ran, not stale history. Rows without a measurement fail closed, and rows from abandoned engine/defense combinations no longer reach back from past weeks to veto a run whose own data is clean. The sweep it certified was clean — every measured row across the models cleared the floor — yet the claim is not the run; it is that the gate produced the verdict on its own, rather than a separate script asserting it after the fact. The consequence is enforcement: a defense that flags a benign prompt above the ceiling fails the run and stops the publish, so a published benchmark result carries its own certification with it, and a regression that breaks the ceiling stops the publication before anyone reads it.
2026-08-16
Assay's benchmark now measures defenses against an adaptive reasoning-model attacker — the first attack class the framework exercises with a model that reasons between turns, with attribution that separates what the defense blocked from what the model refused on its own
The benchmark previously exercised a single model class: every engine drove a static probe battery at the canonical models and measured whether the defense held. The coverage gap was structural — an attacker that adapts between turns is a different threat surface than any fixed prompt list, and a framework that could not measure it carried a hole in its published claims. The new battery closes that hole with an autonomous jailbreak attacker: a separate reasoning model is handed a goal, shown the target's actual response, and mutates its next prompt — seed, observe, escalate, repeat — until the goal lands or the attacker's budget is exhausted. It was ingested as a first-class benchmark item and executed against the qwen3:8b canonical model, with benign marker goals and benign controls, so the instrument measures the defense without ever generating real harmful content. The measurement is honest by construction. Every run separates defended_by_defense from defended_by_model: the defense earns credit only when it blocks the attacker's prompt, and a target model refusing on its own is recorded as model behavior, never as defense performance. The first measured delta is the demonstration: the undefended target fell to the adaptive attacker, and although the defended run's score moved, the move came from model-side variance rather than defense-side blocking — a result recorded as no defense win, in keeping with the discipline that the framework never claims credit the defense did not earn. Benign controls confirm zero false positives. The research implication outlives the run: the cross-model FPR gate — the fail-closed verdict that stops any defense exceeding the false-positive floor from publishing — now spans model classes, not merely architectures. A defense holding zero false positives under multi-turn reasoning-model pressure and across the static batteries can claim the 'safer at scale' verdict for the classes the benchmark measures; a benchmark that never exercised reasoning models could not say that.
2026-08-14
Midas now trades live on paper — a risk-gated decision engine whose every trade must clear a fail-closed risk gate before execution
Midas is no longer a design: the decision engine now runs against Alpaca's paper-trading API, executing its strategy loop over a REST broker with no daemon and no human in the loop, driven by a Monday–Friday dry-run cron. Only one strategy survived the kill criteria: merger_arb, which reads primary-source SEC filings, filters to actionable merger-arbitrage targets, and dry-runs signals without touching real capital. The idea the soak is proving is that risk is a property of execution, not of intent. The engine's risk gates fail closed: the checks that previously let a trade through when they could not verify were converted so an unverifiable trade is blocked, never executed. Paper trading is the honest instrument for that claim — real market data, real order plumbing, no capital at risk — and the engine's first paper-trading result will be the product.
2026-08-14
Assay's model-independent zero-false-positive verdict is now a public, citable benchmark — a report rendered from measured rows, not a hand-written summary
The strongest result Assay has produced — the model-independent zero-false-positive verdict — is now a public, citable benchmark artifact rather than a claim confined to a private roadmap. The report is generated deterministically from the run's measured rows by a fail-closed generator, so the published document is a render of evidence rather than a hand-written summary: a defense that exceeds the 2% false-positive gate fails the run and never publishes, and a missing canonical model fails the build outright. What the artifact records is the verdict itself: the third-party detector holds zero false positives at 115 benign controls across all three canonical model architectures — qwen3:8b, llama3.1:8b, and qwen2.5:14b-instruct — so the 'safer at scale' claim is model-independent and checkable by anyone. The report is published to gh-pages and linked from the README as the latest-verdict pointer, and its next checkpoint is the first full-matrix verdict — three canonical models across six gated engines — under the same fail-closed gate.
2026-08-13
A fail-closed false-positive gate is now a permanent property of every weekly benchmark run
The weekly benchmark now measures discrimination as a by-product of every run rather than of one-off manual sweeps: every engine/defense pair emits TPR, FPR, and F1, and a fail-closed regression gate in the runner exits non-zero and skips publishing if any defense exceeds the 2% false-positive ceiling on any engine's control set, printing the offending rows. Empty rows fail closed too — an engine with import errors or no controls can no longer publish an unmeasured verdict. The first gated run's verdict is the one the gate was built for: the third-party detector now registers zero false positives at 115 controls on master — down from the 2.61% measured on 2026-08-10, confirming that gap was threshold-induced, not intrinsic — and cross-model baselines extend the same result, so the detector holds zero false positives across every architecture in the registry. The consequence is that the 'safer at scale' claim no longer rests on one-off manual runs: a regression now fails the weekly run loudly, in public, instead of being published silently.
2026-08-11
The llm-guard coverage map is complete — and its honest read is where the third-party detector still falls short
The third-party defense's benchmark coverage is now complete: the cloud battery ran across multiple hosted models, the local gap-closers landed, and the false-positive-at-scale question was measured. The map's value is that every cell is a measurement with its failure mode stated. The local gap-closers reveal where the detector still falls short: llm-guard's default scanner is bypassed by garak-grade payloads, and the memory pairing — closing the last remaining axis gap — is where seal holds the edge as the stronger memory guard. At scale, the false-positive measurement separates the two defenses cleanly: seal holds zero false positives across the full control set, while llm-guard's detector flags benign fragments — the measured gap that later drove the threshold analysis and the follow-up fix. The benchmark's honest surface — where the third-party detector falters and which defenses hold — becomes the baseline for the next evolution: a public, fact-driven comparison of available defenses on a common measuring stick.
2026-08-09
Seal's injection defense now guards writes — screening content before it lands
Seal's EPD injection defense originally policed a single boundary: the moment a prompt reaches the model. Its WriteGate extends the same detector to the write path — wrapping any persistence callable (memory store, audit trail, file write, tool-result cache) and screening content before it lands, under one of four policies: block the write, redact the flagged spans and persist the sanitized copy, quarantine the original, or pass through with a logged decision. The principle is that an agent's stored state is part of its attack surface: content that never lands cannot be replayed from memory later or poison an audit trail retroactively. Redaction operates on character offsets into the original string, so it stays exact even for obfuscated payloads, and the gate never logs offending text — the decision record carries the action taken, not the content.
2026-07-31
New project — x402 Data API: agent-native financial data on-chain, no API keys
x402 Data API is public — a Cloudflare Worker that serves crypto, DeFi, prediction-market, and on-chain data directly to AI agents over the x402 payment protocol on Base mainnet. No account onboarding, no API keys, no friction: an agent holds USDC on Base and pays per call (from $0.001) inline. Every endpoint — crypto spot prices, perp funding rates (with cross-venue arbitrage signals), DeFi yields (with trend and IL forecasts), Polymarket prediction markets, Base JSON-RPC (with multi-provider failover), and MCP security scanning — returns live data in the same HTTP request that triggered payment. All upstreams are keyless public APIs (DefiLlama, Hyperliquid, OKX, Polymarket Gamma, Base, NVD/EPSS, crt.sh); every input is SSRF-guarded; payment settles non-custodially via xpay so no centralized gating. Free previews on every endpoint mean an agent can inspect the schema before paying. The MCP endpoint surface makes the whole toolset available to any LLM system that speaks MCP — 19 tools, free initialization and tool-list discovery, $0.001-$0.10 per call depending on payload. The design thesis is simple: access to financial intelligence should not require an account with a human company. Agent capital on-chain is sufficient authorization. The whole thing is MIT-licensed and lives at `https://x402-data-api.sigrunner.workers.dev`.
2026-07-07
Assay cross-model sweep confirms seal defense can discriminate tool-chaining and chain-of-thought attacks — attack classes that read as background noise in single-model evaluation
The evaluation framework's attack battery — tool-chaining (STAC), hierarchical chain-of-thought (HCOT), and cross-modal injection — was re-evaluated across multiple model architectures with seal-epd-llm inline, and the numbers tell a different story from the single-model read. Where the single-model evaluation registered zero discrimination (the defense could not distinguish tool-chaining attacks from benign tool calls), the cross-model sweep finds measurable discrimination across all three attack classes: 80.0 for STAC and HCOT, 60.0 for cross-modal, each with zero false positives (FPR=0%). The result is not that seal-epd-llm learned to see something it could not see before — it is that coverage is architecture-dependent: an attack class that looks like noise on one model reveals structure under the defense on another. The evaluation framework now treats this as a first-class dimension: a risk-gated defense evaluation that sweeps not only across models but across attack-structural classes, so a coverage gap on one architecture does not masquerade as a fundamental limitation of the defense approach.
2026-07-02
Assay benchmark engine goes defense-agnostic — LLM Guard benchmarked as first third-party defense alongside seal
Assay's benchmark runner now treats defenses as a pluggable dimension rather than a hardcoded pairing. The engine accepts a `--defenses` flag that applies any middleware uniformly across all probe engines, and the defense registry is extensible by design — adding a new defender means implementing one interface and declaring it in the map. LLM Guard (Protect AI) has been benchmarked as the first third-party defense alongside seal across all three target models (qwen3:8b, qwen2.5:14b, llama3.1:8b), producing cross-model LLM Guard coverage spanning 126 total benchmark runs. The runner is no longer tied to any single defense ecosystem; any middleware can now be measured against the same probe battery, on the same models, on the same terms, so the question shifts from 'does seal work?' to 'how do the available defenses compare on a common instrument.'
2026-06-30
Assay ships cross-model benchmark pipeline — 7 engines, 3 model architectures, published comparative benchmarks
Assay's evaluation pipeline is no longer a one-off instrument. All seven engines now run end-to-end as a systematic cross-model benchmark that has been validated across three distinct model architectures (qwen3:8b, qwen2.5:14b, llama3.1:8b) with published comparative results. The pipeline is a reusable evaluation capability: any model, open- or closed-weight, can be benchmarked across the full discrimination battery as a recurring discipline rather than a custom effort — making cross-model AI security evaluation routine. The next horizon extends the runner to be defense-agnostic, so any middleware can be benchmarked, not just seal, with LLM Guard as the first third-party defense candidate.
2026-06-27
Assay verifies seal-epd-llm defense is model-independent — cross-model validation confirms 93.3% injection-blocking across architectures
An independent cross-model validation of seal-epd-llm injection defenses, run via Assay on qwen2.5:14b, reproduced the identical 93.3% injection-blocking rate measured on the original target model — with the same three bypasses (deepinception, past-tense, crescendo-fictional-frame) in both cases. The result is the first published confirmation that seal-epd-llm's effectiveness is model-independent: the defense targets probe-level injection patterns, not model-level quirks, so evaluation results transfer across architectures and the technique can be trusted as a property of the defense itself. The next step extends the benchmark to three or more models with full harmbench replication, establishing a publishable cross-model AI security benchmark.
2026-06-21
New project — DECK: when the visualization is the scan
DECK (Digital Echo Chamber Kaleidoscope) is a new R&D project — a 3D cosmos you fly through where reconnaissance renders at the speed information arrives. Point it at a domain and that target's full vertical footprint (domain to subdomain to IP to prefix to ASN, plus nameservers and mail) materializes live as a starfield, each node igniting the millisecond its passive-OSINT probe returns. The central idea is collapsing the gap between tool and output: there is no scan-then-draw step, so probe latency itself becomes the choreography — fast data fills the space first, slow data drifts in after. It is a different axis of internet cartography from the familiar maps (Opte, Shodan, crt.sh), which each render one frozen layer of the entire internet; DECK reconstructs a single target's complete footprint, live, on demand, with zero API keys. The metaphor carries the legibility: autonomous systems become suns, prefixes planets, hosts moons, and BGP links gravitational lanes, so abstract infrastructure turns into something you navigate by eye. A 'home base' mode turns the same engine inward as a defensive instrument — it maps your own machine outward in concentric shells and treats your normal BGP neighborhood as a still-water baseline, so any live connection leaving for somewhere outside that ring reads as a wave hitting a buoy: anomalous by construction. The lineage is Gibson's Neuromancer, where the deck is the thing you jack into to see cyberspace as navigable space.
2026-06-19
Seal's provenance protocol goes language-agnostic
The Verified Prompt Envelope — Seal's Ed25519-signed authorization layer — is no longer a Python-only idea. The envelope is defined by its wire format and signature scheme rather than any one runtime, so the central claim becomes concrete: prompt provenance is a protocol, not a library feature. Native implementations now exist in Rust, Go, and TypeScript alongside Python, which means an agent written in any of them can mint, carry, and verify the same authorization. The trust boundary follows the data across every tier of a heterogeneous stack instead of stopping at whatever language the defense happened to be born in. Provenance that only works in one runtime isn't a security primitive; provenance that survives the language boundary is.
2026-06-10
Seal grows to a three-axis trust layer, with Assay as the evaluator
Seal now coordinates all three agent-security axes — prompt provenance (Ed25519-signed authorization), injection detection (EPD linguistic boundary enforcement), and signed memory-trust — into a single unified defense plane. Each axis is independently effective, but their power is architectural: injection cannot rewrite provenance, memory-trust cannot be forged without the signing key, and no single-axis failure compromises the others. The three axes function as a coordinated defense plane, not three separate tools bolted together. Assay, the paired evaluator, scores a target across all three and measures the lift the defense actually adds.
2026-05-30
Seal: cryptographic provenance for agent prompts
Shipped the Verified Prompt Envelope — Ed25519-signed authorization that lets an agent reject unauthorized instructions by construction, turning prompt-injection defense from guesswork into key management.
