Field completeness · reproducibility artifact

The counts behind the chart, so you can rebuild it without us.

A chart published without its numerators is an assertion. This page carries the denominator and every count for each year, the extraction query that produced them, and the script that turns them into the picture. Measured once, at 2026-09-10 20:44:52 UTC.

What the denominator is, before any number below

These are completeness rates for our extraction, not for the public record.

The denominator is rows Veridion collected and admitted to its serving path. It already excludes anything our pipeline failed to collect or failed to admit. A populated field here cannot establish what the filings contain, and it cannot establish what a different parser would read.

A blank field is one of two things and these counts cannot separate them: the field was blank on the filed document, or our parser did not read it. Owner type is optional in places on the House form.

No cause is assigned to any movement in these series. Field presence can shift with asset mix, chamber mix, collection coverage and parser changes, and this data separates none of them. A rise is not evidence that anything improved.

Section 1

Counts, not rates

Every percentage on the chart is computed from this table. Nothing is hand-typed as a rate, on the chart or on this page.

Transaction yearRows (denominator)TickerOwner typeAll four
20142,1251,315889664
20153,1522,1791,8721,423
20163,3932,1992,0211,439
20173,3942,1781,8241,333
20183,8262,3582,2401,609
20195,1093,1402,4331,564
20205,3553,5962,9932,030
20214,5373,4132,6341,938
20223,6382,8982,2171,804
20234,4564,0482,3982,146
20242,9172,5931,9951,810
20258,0147,2914,3263,867
Total49,916———

“All four” means ticker symbol, owner type, a disclosed amount floor and asset name all populated on the same row. The four columns are not additive: a row counted under ticker may also be counted under owner.

Amount is tested on the lower bound only, and that is deliberate rather than lax. The top disclosure band is “Over $50,000,000”, which has no upper bound on the form, so requiring one would penalise a correctly completed filing. Measured on this subset: 49,916 rows carry an amount floor, 49,913 also carry an upper bound, and the 3 that do not include the 2 rows in that open-ended band. Requiring both bounds would move the “all four” counts by at most 3 rows across twelve years.

Query that produced this table
with adm as (
  select *, extract(year from transaction_date)::int as yr
  from warehouse
  where source in ('pdf', 'official-senate-efd')
  and superseded_at is null
  and nullif(trim(member_slug), '') is not null
  and nullif(trim(member), '')      is not null
  and nullif(trim(doc_id), '')      is not null
  and transaction_date is not null
  and filing_date      is not null
  and filing_date >= transaction_date
  and pdf_url ~ '^https://(disclosures-clerk[.]house[.]gov/.+[.]pdf([?].*)?|efdsearch[.]senate[.]gov/.+)$'
)
select yr as year,
  count(*) as rows_denominator,
  count(*) filter (where nullif(trim(coalesce(ticker, '')), '') is not null) as ticker_n,
  count(*) filter (where owner_type is not null) as owner_n,
  count(*) filter (where nullif(trim(coalesce(ticker, '')), '') is not null
                     and owner_type is not null
                     and amount_low is not null
                     and nullif(trim(coalesce(asset, '')), '') is not null) as all_four_n
from adm
where yr between 2014 and 2025
group by yr order by yr;
Query for the amount-bound detail
-- "All four" tests the amount floor, not a complete range. The top band is
-- "Over $50,000,000" and has no upper bound on the form, so requiring one
-- would penalise a correctly completed filing.
with adm as (
  select *, extract(year from transaction_date)::int as yr
  from warehouse
  where source in ('pdf', 'official-senate-efd')
  and superseded_at is null
  and nullif(trim(member_slug), '') is not null
  and nullif(trim(member), '')      is not null
  and nullif(trim(doc_id), '')      is not null
  and transaction_date is not null
  and filing_date      is not null
  and filing_date >= transaction_date
  and pdf_url ~ '^https://(disclosures-clerk[.]house[.]gov/.+[.]pdf([?].*)?|efdsearch[.]senate[.]gov/.+)$'
), sub as (select * from adm where yr between 2014 and 2025)
select count(*) filter (where amount_low is not null)                             as floor_present,  -- 49,916
       count(*) filter (where amount_low is not null and amount_high is not null) as both_bounds,    -- 49,913
       count(*) filter (where amount_range = 'Over $50,000,000')                  as open_ended      --      2
from sub;

Section 2

Why documents and filers do not sum

This is the part a careful reader will try to check and fail to reconcile, so it is stated before the numbers rather than after.

Rows

49,916

Sums from the yearly table above.

Distinct documents

6,133

Yearly counts sum to 6,310.

Distinct filers

351

Yearly counts sum to 1,274.

One document can hold transactions in more than one year, and one filer files across many years, so the distinct counts are smaller than the yearly sums. Rows are the only column that adds up. Quoting a document or filer figure derived by addition would be wrong by roughly 3% and 260% respectively.

Query for the subset totals
-- Distinct counts over the whole 2014-2025 subset. These deliberately do
-- NOT equal the sum of the yearly counts: one document and one filer can
-- appear in more than one transaction year.
with adm as (
  select *, extract(year from transaction_date)::int as yr
  from warehouse
  where source in ('pdf', 'official-senate-efd')
  and superseded_at is null
  and nullif(trim(member_slug), '') is not null
  and nullif(trim(member), '')      is not null
  and nullif(trim(doc_id), '')      is not null
  and transaction_date is not null
  and filing_date      is not null
  and filing_date >= transaction_date
  and pdf_url ~ '^https://(disclosures-clerk[.]house[.]gov/.+[.]pdf([?].*)?|efdsearch[.]senate[.]gov/.+)$'
), sub as (select * from adm where yr between 2014 and 2025)
select count(*)                        as rows,          -- 49,916
       count(distinct doc_id)          as documents,     -- 6,133
       count(distinct member_slug)     as filers         -- 351
from sub;

-- The same numbers summed per year instead, which is the wrong way to get
-- them and is published here so the difference is checkable rather than
-- asserted: 6,310 documents and 1,274 filers.
with adm as (
  select *, extract(year from transaction_date)::int as yr
  from warehouse
  where source in ('pdf', 'official-senate-efd')
  and superseded_at is null
  and nullif(trim(member_slug), '') is not null
  and nullif(trim(member), '')      is not null
  and nullif(trim(doc_id), '')      is not null
  and transaction_date is not null
  and filing_date      is not null
  and filing_date >= transaction_date
  and pdf_url ~ '^https://(disclosures-clerk[.]house[.]gov/.+[.]pdf([?].*)?|efdsearch[.]senate[.]gov/.+)$'
), sub as (select * from adm where yr between 2014 and 2025)
select sum(d) as sum_yearly_documents,   -- 6,310
       sum(f) as sum_yearly_filers       -- 1,274
from (select count(distinct doc_id) d, count(distinct member_slug) f
      from sub group by yr) per_year;

Section 3

The admission predicate, verbatim

The rule as written, not a paraphrase. Note the last line: the document URL is matched by pattern. We do not fetch it, so nothing here claims that any document currently resolves.

Serving admission
where source in ('pdf', 'official-senate-efd')
  and superseded_at is null
  and nullif(trim(member_slug), '') is not null
  and nullif(trim(member), '')      is not null
  and nullif(trim(doc_id), '')      is not null
  and transaction_date is not null
  and filing_date      is not null
  and filing_date >= transaction_date
  and pdf_url ~ '^https://(disclosures-clerk[.]house[.]gov/.+[.]pdf([?].*)?|efdsearch[.]senate[.]gov/.+)$'

Queries are printed against the alias warehouse. The physical relation name is withheld from public pages by a build gate — the same gate that keeps any row we did not collect ourselves out of these counts. The predicate is what you need to check the logic; the name is not.

Section 4

The plotting script

The script carries the counts, asserts that they sum and that no numerator exceeds its denominator, and derives every rate. Running it reproduces the published image exactly.

plot_field_completeness.py
# plot_field_completeness.py (excerpt) - the full script is linked below.
# Every rate on the chart is computed here from the counts above. No
# percentage is hand-typed anywhere in the pipeline.

COUNTS = {  # year: (rows_denominator, ticker_n, owner_n, all_four_n)
    2014: (2125, 1315,  889,  664),
    2015: (3152, 2179, 1872, 1423),
    ...
    2025: (8014, 7291, 4326, 3867),
}

assert sum(v[0] for v in COUNTS.values()) == 49_916
for year, (denom, *nums) in COUNTS.items():
    for n in nums:
        assert 0 <= n <= denom          # a numerator cannot exceed its denominator

rate = lambda idx: [100.0 * COUNTS[y][idx] / COUNTS[y][0] for y in sorted(COUNTS)]

The complete script, including the embedded query and every count: plot_field_completeness.py. Save it, run it with matplotlib installed, and it writes the published PNG.

What this is not

Not a measure of disclosure quality, not a compliance finding, and not an accusation. It counts which fields our parser populated. A filer whose row shows a blank owner type may have completed the form correctly; the gap may be entirely ours. That ambiguity is the honest state of the measurement and we are not going to resolve it by choosing the flattering reading.

Weekly Veridion brief

Rating changes, public disclosure activity, methodology notes, and product updates. One email per week. No advertising list resale.