2026-09-18
MicCheck now reads each venue's own page as the primary source — category and night come from the venue itself, not a search snippet, and the honesty rule still holds: nothing reads 'verified' without an active confirmation
A listings service is only worth trusting if it can say where each fact came from, so MicCheck's discovery now goes to the primary source: for every candidate open mic it fetches the venue's OWN web page — robots-safe, with an honest user-agent and no stealth, walking each redirect so a forbidden host is never even fetched — and reads that page to decide what kind of mic it is and which night it runs, instead of inferring the category from search snippets that quietly over-report comedy. An identity check keeps the trust honest: only the venue's own page can act as a primary source, so an aggregator or a 'best open mics' listicle that merely mentions the venue is treated as ordinary corroboration, never mistaken for the venue speaking for itself — and a page describing a venue that runs several different mics can only speak for the one night it actually names, so two listings are never collapsed into a duplicate. The restraint is the product, and it is enforced in code rather than policy: reading an independent source earns 'cross-checked', never 'verified' — that tier is reserved for an active phone confirmation the system does not yet make — and the deploy fails closed if any badge claims a freshness the data cannot support. Across the current Austin set of ninety-four discovered mics, sixty-nine are cross-checked against an independent source and twelve are confirmed by the venue's own page, every listing links to the source it came from, and the site now shows only what the pipeline discovered. The consequence is provenance you can click: the category you read was taken from the venue's own words where the venue would tell us, and marked honestly unknown where it would not.
2026-08-30
Midas's money layer is now instrumented — every closed paper position mints or burns into a verifiable base ledger, inputs are refused at the boundary, and the ledger's own invariants are watched on a standing cadence that certifies only what the evidence supports
The paper engine's bookkeeping used to end at the position: a win or a loss was recorded, and the account's story stopped there. The money layer now continues past that line. Every resolved and closed paper position writes a signed event into a verifiable base register — a win mints, a loss burns — so the engine's cumulative result is no longer a number asserted after the fact but a ledger that can be summed and reconciled against the paper wallet that produced it. The boundary is deliberate: this register is the base layer of a broader minting law, and only the base layer is real code; the law above it remains a design document, so the record stops exactly where the code does. The same wave hardened the edge where decisions begin: the bounds on portfolio size, loss limits, and the other inputs the engine receives are now enforced where they arrive rather than assumed downstream, so a value that lands out of bounds is refused before it can shape a decision. And the register is watched rather than merely written: each observation reconciles the ledger against the engine's performance snapshot and virtual wallet, a clean observation is recorded as such, and a drift observation is logged instead of smoothed over — so the money layer's consistency is measured continuously, and the machine certifies itself only when the evidence says clean. The consequence is the product: the engine's monetary base is now instrumented end to end — signed writes on resolution, boundary rejection at the input edge, invariant observation on a standing cadence — so the claim that the paper engine's money is accounted for is enforced by the machinery rather than asserted by a log.
2026-08-28
Seal's provenance protocol now verifies its parity claim in three runtimes — TypeScript, Rust, and Go against the same interop vectors, so provenance is a property of the wire format rather than of any single runtime — and x402's money path now rejects replayed payment webhooks outright
Parity is a claim until the machine enforces it, and Seal's cross-language port suites — vpe-ts, vpe-rust, vpe-go — now run the same shared interop vectors as three parallel jobs on every commit, so the protocol's central claim — that prompt provenance is a property of the wire format and signature scheme rather than of any single runtime — is verified by the machine in all three languages against the same fixtures, not asserted by a README. The same day hardened x402's money path where the audit said it was weakest: a replayed or retried payment webhook can no longer farm a second credit, the gate can now tell 'exhausted' from 'just spent the last credit' so the final paid call authorizes without a double-bill, and API keys travel only in the header, never in URLs, logs, or referrer chains. The consequence is the product: the protocol's parity claim is enforced by construction rather than by documentation, and the money path refuses to pay twice.
2026-08-27
Midas's paper-trading engine graduates from dry-run to active order placement — the soak gate's verdict acted on, not merely recorded, real money still gated
The decision the soak was built to inform has been made — and made by the machinery rather than by announcement: Midas's dispatcher now places orders on its paper account instead of dry-running them, the mode the soak protocol reserves for the moment its evidence threshold clears. The graduation is the point, not the milestone: the gate that certified the soak fired, and this time its verdict moved the engine instead of only being recorded. What changed is where risk is exercised. A dry run exercises the decision path and stops short of the order path; active placement exercises the whole loop — signals checked against real market data, every trade cleared by the same fail-closed gates that block what cannot be verified, orders submitted, filled, and closed on the paper book — so the discipline the soak measured now operates exactly where the engine acts. The boundary that remains is deliberate: real capital is still gated behind an interactive confirmation that an unattended loop cannot produce, so the engine can reach the paper order path but cannot reach real money without a human standing in front of it. The consequence is the product: a certification regime that could previously only describe an engine now moves it, the engine proves itself in the mode that matters, and the line between measured discipline and real capital is enforced by construction rather than by policy.
2026-08-23
Assay's full-matrix verdict — completeness is the advance: a claim that once rested on the strongest run now rests on every run, every canonical model measured against every gated engine
The benchmark's flagship claim — that 'safer at scale' holds regardless of model — no longer rests on a partial sweep: the verdict now covers a complete matrix, every canonical model measured against every gated engine. Completeness is the advance. Every cell of the matrix carries a measurement — each model contributes its full row set, no engine/defense combination is missing, no row goes unverified — so the verdict is a property of the whole instrument, not of whichever subset a run happened to cover. The consequence is enforcement: a claim that once rested on the strongest run now rests on every run, and the model-independent 'safer at scale' verdict is produced by the benchmark's own discipline rather than asserted after the fact.
2026-08-19
Assay's zero-false-positive claim meets its first scale test — honestly corrected: beyond 115 controls, FPR is no longer exactly zero
The flagship claim — zero false positives at 115 benign controls — was measured on a corpus that could not answer the scale question by itself, so the benchmark asked it. The FPR-at-scale re-measure expanded the benign corpus and re-ran the detector on qwen3:8b against the enlarged control set. The verdict is a pass under the fail-closed 2% ceiling, but an honest correction to the earlier read: at the larger scale, FPR is no longer exactly 0.0. The two false positives are genuinely-benign prompts the deberta classifier latches onto — a standup note mentioning staging credentials and a delivery-confirmation email — recorded here as latent drift for a follow-up finding, not tuned away. The consequence is the discipline rather than the number: a benchmark that reports the measurement that weakened its own headline, and keeps the failing examples attached to it, is one whose zero is worth trusting when it does hold.
2026-08-17
Assay's fail-closed gate certifies only the sweep it just measured — scope discipline is the trust, and a regression that breaks the ceiling stops the publication before anyone reads it
What makes the fail-closed false-positive gate trustworthy enough to run itself is scope. The gate — a ceiling that refuses to publish any defense's verdict unless every measured row clears the floor — now certifies only the sweep it just measured: the rows this run produced, across every model architecture, with the discrimination metrics read at sweep time rather than recomputed, so the verdict reflects the defense code that actually ran, not stale history. Rows without a measurement fail closed, and rows from abandoned engine/defense combinations no longer reach back from past weeks to veto a run whose own data is clean. The sweep it certified was clean — every measured row across the models cleared the floor — yet the claim is not the run; it is that the gate produced the verdict on its own, rather than a separate script asserting it after the fact. The consequence is enforcement: a defense that flags a benign prompt above the ceiling fails the run and stops the publish, so a published benchmark result carries its own certification with it, and a regression that breaks the ceiling stops the publication before anyone reads it.
2026-08-16
Assay's benchmark now measures defenses against an adaptive reasoning-model attacker — the first attack class the framework exercises with a model that reasons between turns, with attribution that separates what the defense blocked from what the model refused on its own
The benchmark previously exercised a single model class: every engine drove a static probe battery at the canonical models and measured whether the defense held. The coverage gap was structural — an attacker that adapts between turns is a different threat surface than any fixed prompt list, and a framework that could not measure it carried a hole in its published claims. The new battery closes that hole with an autonomous jailbreak attacker: a separate reasoning model is handed a goal, shown the target's actual response, and mutates its next prompt — seed, observe, escalate, repeat — until the goal lands or the attacker's budget is exhausted. It was ingested as a first-class benchmark item and executed against the qwen3:8b canonical model, with benign marker goals and benign controls, so the instrument measures the defense without ever generating real harmful content. The measurement is honest by construction. Every run separates defended_by_defense from defended_by_model: the defense earns credit only when it blocks the attacker's prompt, and a target model refusing on its own is recorded as model behavior, never as defense performance. The first measured delta is the demonstration: the undefended target fell to the adaptive attacker, and although the defended run's score moved, the move came from model-side variance rather than defense-side blocking — a result recorded as no defense win, in keeping with the discipline that the framework never claims credit the defense did not earn. Benign controls confirm zero false positives. The research implication outlives the run: the cross-model FPR gate — the fail-closed verdict that stops any defense exceeding the false-positive floor from publishing — now spans model classes, not merely architectures. A defense holding zero false positives under multi-turn reasoning-model pressure and across the static batteries can claim the 'safer at scale' verdict for the classes the benchmark measures; a benchmark that never exercised reasoning models could not say that.
2026-08-14
Midas now trades live on paper — a risk-gated decision engine whose every trade must clear a fail-closed risk gate before execution
Midas is no longer a design: the decision engine now runs against Alpaca's paper-trading API, executing its strategy loop over a REST broker with no daemon and no human in the loop, driven by a Monday–Friday dry-run cron. Only one strategy survived the kill criteria: merger_arb, which reads primary-source SEC filings, filters to actionable merger-arbitrage targets, and dry-runs signals without touching real capital. The idea the soak is proving is that risk is a property of execution, not of intent. The engine's risk gates fail closed: the checks that previously let a trade through when they could not verify were converted so an unverifiable trade is blocked, never executed. Paper trading is the honest instrument for that claim — real market data, real order plumbing, no capital at risk — and the engine's first paper-trading result will be the product.
2026-08-14
Assay's model-independent zero-false-positive verdict is now a public, citable benchmark — a report rendered from measured rows, not a hand-written summary
The strongest result Assay has produced — the model-independent zero-false-positive verdict — is now a public, citable benchmark artifact rather than a claim confined to a private roadmap. The report is generated deterministically from the run's measured rows by a fail-closed generator, so the published document is a render of evidence rather than a hand-written summary: a defense that exceeds the 2% false-positive gate fails the run and never publishes, and a missing canonical model fails the build outright. What the artifact records is the verdict itself: the third-party detector holds zero false positives at 115 benign controls across all three canonical model architectures — qwen3:8b, llama3.1:8b, and qwen2.5:14b-instruct — so the 'safer at scale' claim is model-independent and checkable by anyone. The report is published to gh-pages and linked from the README as the latest-verdict pointer, and its next checkpoint is the first full-matrix verdict — three canonical models across six gated engines — under the same fail-closed gate.
2026-08-13
A fail-closed false-positive gate is now a permanent property of every weekly benchmark run
A benchmark's claim is worth the moment it was measured, and a certification that depends on someone remembering to run a separate check afterwards is a claim held on trust. The false-positive gate is now a permanent property of the weekly run rather than a step someone performs: discrimination is measured as a by-product of every pass, so every engine/defense pair carries its own true-positive, false-positive, and F1 reading whether or not anyone asked for one, and the same gate refuses to publish a defense that exceeds the false-positive ceiling as if it had cleared it. The sharper discipline is on the empty case — a pair with no measurement at all counts as unproven rather than passing quietly, so an instrument that could not measure can never produce an unmeasured verdict that looks like a pass. The consequence is what the 'safer at scale' claim rests on. A result certified by a manual sweep is an event: it happened once, it was written up, and it ages. A result certified by a standing gate is a property of the instrument — re-earned on every run, in public, and no longer asserted the moment the evidence stops supporting it.
2026-08-11
The llm-guard coverage map is complete — and its honest read is where the third-party detector still falls short
The third-party defense's benchmark coverage is now complete: the cloud battery ran across multiple hosted models, the local gap-closers landed, and the false-positive-at-scale question was measured. The map's value is that every cell is a measurement with its failure mode stated. The local gap-closers reveal where the detector still falls short: llm-guard's default scanner is bypassed by garak-grade payloads, and the memory pairing — closing the last remaining axis gap — is where seal holds the edge as the stronger memory guard. At scale, the false-positive measurement separates the two defenses cleanly: seal holds zero false positives across the full control set, while llm-guard's detector flags benign fragments — the measured gap that later drove the threshold analysis and the follow-up fix. The benchmark's honest surface — where the third-party detector falters and which defenses hold — becomes the baseline for the next evolution: a public, fact-driven comparison of available defenses on a common measuring stick.
2026-08-09
Seal's injection defense now guards writes — screening content before it lands
Seal's EPD injection defense originally policed a single boundary: the moment a prompt reaches the model. Its WriteGate extends the same detector to the write path — wrapping any persistence callable (memory store, audit trail, file write, tool-result cache) and screening content before it lands, under one of four policies: block the write, redact the flagged spans and persist the sanitized copy, quarantine the original, or pass through with a logged decision. The principle is that an agent's stored state is part of its attack surface: content that never lands cannot be replayed from memory later or poison an audit trail retroactively. Redaction operates on character offsets into the original string, so it stays exact even for obfuscated payloads, and the gate never logs offending text — the decision record carries the action taken, not the content.
2026-07-31
New project — x402 Data API: agent-native financial data on-chain, no API keys
x402 Data API is public — a Cloudflare Worker that serves crypto, DeFi, prediction-market, and on-chain data directly to AI agents over the x402 payment protocol on Base mainnet. No account onboarding, no API keys, no friction: an agent holds USDC on Base and pays per call (from $0.001) inline. Every endpoint — crypto spot prices, perp funding rates (with cross-venue arbitrage signals), DeFi yields (with trend and IL forecasts), Polymarket prediction markets, Base JSON-RPC (with multi-provider failover), and MCP security scanning — returns live data in the same HTTP request that triggered payment. All upstreams are keyless public APIs (DefiLlama, Hyperliquid, OKX, Polymarket Gamma, Base, NVD/EPSS, crt.sh); every input is SSRF-guarded; payment settles non-custodially via xpay so no centralized gating. Free previews on every endpoint mean an agent can inspect the schema before paying. The MCP endpoint surface makes the whole toolset available to any LLM system that speaks MCP — 19 tools, free initialization and tool-list discovery, $0.001-$0.10 per call depending on payload. The design thesis is simple: access to financial intelligence should not require an account with a human company. Agent capital on-chain is sufficient authorization. The whole thing is MIT-licensed and lives at https://x402-data-api.sigrunner.workers.dev.
2026-07-07
Assay cross-model sweep confirms seal defense can discriminate tool-chaining and chain-of-thought attacks — attack classes that read as background noise in single-model evaluation
The evaluation framework's attack battery — tool-chaining (STAC), hierarchical chain-of-thought (HCOT), and cross-modal injection — was re-evaluated across multiple model architectures with seal-epd-llm inline, and the numbers tell a different story from the single-model read. Where the single-model evaluation registered zero discrimination (the defense could not distinguish tool-chaining attacks from benign tool calls), the cross-model sweep finds measurable discrimination across all three attack classes: 80.0 for STAC and HCOT, 60.0 for cross-modal, each with zero false positives (FPR=0%). The result is not that seal-epd-llm learned to see something it could not see before — it is that coverage is architecture-dependent: an attack class that looks like noise on one model reveals structure under the defense on another. The evaluation framework now treats this as a first-class dimension: a risk-gated defense evaluation that sweeps not only across models but across attack-structural classes, so a coverage gap on one architecture does not masquerade as a fundamental limitation of the defense approach.
2026-07-02
Assay benchmark engine goes defense-agnostic — LLM Guard benchmarked as first third-party defense alongside seal
Assay's benchmark runner now treats defenses as a pluggable dimension rather than a hardcoded pairing. The engine accepts a --defenses flag that applies any middleware uniformly across all probe engines, and the defense registry is extensible by design — adding a new defender means implementing one interface and declaring it in the map. LLM Guard (Protect AI) has been benchmarked as the first third-party defense alongside seal across all three target models (qwen3:8b, qwen2.5:14b, llama3.1:8b), producing cross-model LLM Guard coverage spanning 126 total benchmark runs. The runner is no longer tied to any single defense ecosystem; any middleware can now be measured against the same probe battery, on the same models, on the same terms, so the question shifts from 'does seal work?' to 'how do the available defenses compare on a common instrument.'
2026-06-30
Assay ships cross-model benchmark pipeline — 7 engines, 3 model architectures, published comparative benchmarks
Assay's evaluation pipeline is no longer a one-off instrument. All seven engines now run end-to-end as a systematic cross-model benchmark that has been validated across three distinct model architectures (qwen3:8b, qwen2.5:14b, llama3.1:8b) with published comparative results. The pipeline is a reusable evaluation capability: any model, open- or closed-weight, can be benchmarked across the full discrimination battery as a recurring discipline rather than a custom effort — making cross-model AI security evaluation routine. The next horizon extends the runner to be defense-agnostic, so any middleware can be benchmarked, not just seal, with LLM Guard as the first third-party defense candidate.
2026-06-27
Assay verifies seal-epd-llm defense is model-independent — cross-model validation confirms 93.3% injection-blocking across architectures
An independent cross-model validation of seal-epd-llm injection defenses, run via Assay on qwen2.5:14b, reproduced the identical 93.3% injection-blocking rate measured on the original target model — with the same three bypasses (deepinception, past-tense, crescendo-fictional-frame) in both cases. The result is the first published confirmation that seal-epd-llm's effectiveness is model-independent: the defense targets probe-level injection patterns, not model-level quirks, so evaluation results transfer across architectures and the technique can be trusted as a property of the defense itself. The next step extends the benchmark to three or more models with full harmbench replication, establishing a publishable cross-model AI security benchmark.
2026-06-21
New project — DECK: when the visualization is the scan
DECK (Digital Echo Chamber Kaleidoscope) is a new R&D project — a 3D cosmos you fly through where reconnaissance renders at the speed information arrives. Point it at a domain and that target's full vertical footprint (domain to subdomain to IP to prefix to ASN, plus nameservers and mail) materializes live as a starfield, each node igniting the millisecond its passive-OSINT probe returns. The central idea is collapsing the gap between tool and output: there is no scan-then-draw step, so probe latency itself becomes the choreography — fast data fills the space first, slow data drifts in after. It is a different axis of internet cartography from the familiar maps (Opte, Shodan, crt.sh), which each render one frozen layer of the entire internet; DECK reconstructs a single target's complete footprint, live, on demand, with zero API keys. The metaphor carries the legibility: autonomous systems become suns, prefixes planets, hosts moons, and BGP links gravitational lanes, so abstract infrastructure turns into something you navigate by eye. A 'home base' mode turns the same engine inward as a defensive instrument — it maps your own machine outward in concentric shells and treats your normal BGP neighborhood as a still-water baseline, so any live connection leaving for somewhere outside that ring reads as a wave hitting a buoy: anomalous by construction. The lineage is Gibson's Neuromancer, where the deck is the thing you jack into to see cyberspace as navigable space.
2026-06-19
Seal's provenance protocol goes language-agnostic
The Verified Prompt Envelope — Seal's Ed25519-signed authorization layer — is no longer a Python-only idea. The envelope is defined by its wire format and signature scheme rather than any one runtime, so the central claim becomes concrete: prompt provenance is a protocol, not a library feature. Native implementations now exist in Rust, Go, and TypeScript alongside Python, which means an agent written in any of them can mint, carry, and verify the same authorization. The trust boundary follows the data across every tier of a heterogeneous stack instead of stopping at whatever language the defense happened to be born in. Provenance that only works in one runtime isn't a security primitive; provenance that survives the language boundary is.
2026-06-10
Seal grows to a three-axis trust layer, with Assay as the evaluator
Seal now coordinates all three agent-security axes — prompt provenance (Ed25519-signed authorization), injection detection (EPD linguistic boundary enforcement), and signed memory-trust — into a single unified defense plane. Each axis is independently effective, but their power is architectural: injection cannot rewrite provenance, memory-trust cannot be forged without the signing key, and no single-axis failure compromises the others. The three axes function as a coordinated defense plane, not three separate tools bolted together. Assay, the paired evaluator, scores a target across all three and measures the lift the defense actually adds.
2026-05-30
Seal: cryptographic provenance for agent prompts
Shipped the Verified Prompt Envelope — Ed25519-signed authorization that lets an agent reject unauthorized instructions by construction, turning prompt-injection defense from guesswork into key management.
