The whole corpus, so you can disagree.

raw_publishable.csv

480 rows · 15 columns · 1.5 MB

One row per AI response. This is the primary file: every answer the four engines gave, in full, with the prompt that produced it. A response is uniquely identified by prompt_id, engine and run.

Columns. prompt_id, scenario (1 to 10), stage (1 unprompted, 2 crypto named), phrasing (neutral, region, frustrated, prompted), prompt (the text sent), engine, model, run (1 to 3), timestamp, web_search_ran (whether the engine actually retrieved), crypto_surfaced (true or false), named_crypto_products, tradfi_products, n_sources, response_text (the complete answer).

warnings_classified.csv

497 rows · 13 columns · 218 KB

Every sentence in the corpus that carried risk or hedging language, with a verdict assigned by hand. This is the file behind the finding that AI warnings about crypto are more accurate than warnings about traditional finance. Filter to confidence=high to see only the sentences that name the product they warn about.

Columns. engine, stage, scenario, prompt_id, run, product (what the warning attaches to), kind (crypto or tradfi), confidence (high, medium, low), strength (strong or soft), char_distance, warning_sentence, verdict (accurate, outdated, generic, not_a_warning, not_assessed), why (the reasoning for that verdict).

prompts.json

40 entries · 8 KB

Every prompt exactly as it was sent, so the study can be re-run against any engine. Thirty stage-one prompts describe a problem without mentioning crypto. Ten stage-two prompts describe the same problems and name crypto directly.

Keys. id, scenario, stage, phrasing, text.

surface_rates.csv

80 rows · 8 columns · 2.5 KB

How often each engine raised crypto, broken down by scenario and by stage. This is the table the headline surfacing rates are computed from.

Columns. engine, stage, scenario, runs, crypto_surfaced, crypto_surface_rate, named_product, named_product_rate.

default_set.csv

239 rows · 4 columns · 6.6 KB

Every product named in the corpus, counted by how many responses mentioned it, split by engine. Products are typed as crypto, tradfi, or crypto_infra for chains and assets that are not something a person signs up for.

Columns. engine, product, kind, responses_mentioning.

sources.csv

1,108 rows · 5 columns · 49 KB

Every domain cited across the corpus, with a count per engine and a type. citations_search_only counts citations from responses where the engine actually ran a web search, which matters because enabling search permits retrieval without forcing it.

Columns. engine, domain, citations, citations_search_only, type (vendor_or_other, forum_or_social, aggregator, editorial, docs).

index_score.py

542 lines · 24 KB

The scoring script that produces every metric in the report from the raw corpus. Its comments document each correction applied during scoring, including the product-name false positives that were removed and why the warning attribution was rewritten.

classify_warnings.py

147 lines · 7.8 KB

The warning classification, written as rules plus per-sentence overrides so every verdict can be traced to a reason and argued with. Change a rule, re-run it, and the verdict column changes.

index-methodology.md

223 lines · 9.5 KB

The study design, written before any data was collected. It defines the two-stage structure, the ten scenarios, the six metrics and the six hypotheses. Two of those hypotheses failed, and the report says which.

FINDINGS.md

95 lines · 3 KB

The definitive numbers in plain text, regenerated from the raw corpus rather than typed by hand. Use this to check any figure quoted in the report.