The CHIRPdb dataset

What you have

Read access to a running PostgreSQL database, the source report PDFs, and the paperwork to make sense of both. No access to CHIRP's application, pipeline or APIs; everything you need is in the four pieces below.

Working with the database

It is ordinary PostgreSQL - two things worth knowing before you query:

• Every table is readable. All tables allow public read; there is nothing to unlock first.
• tsvector columns are generated. The full-text search columns are computed automatically from their source text and are read-only - query them for search, don't write them.

What the dataset is

Each published report (a PDF, from MAIB or NTSB) is broken into sentence-level text plus the report's own stated incident particulars. MAIB reports are additionally linked to structured incident data - occurrences, vessels, affected persons - drawn from the MAIB Data Portal spreadsheets; NTSB has no equivalent spreadsheet source, so that layer is MAIB-only. An analysis pipeline then extracts findings from the text: safety issues, recommendations, named entities, and causal-factor classifications from CHIRP's SHIELD taxonomy. Every extracted finding is traceable back to the exact sentences of the source report.

documents.jurisdiction separates the two corpora: 'UK' for the 796 MAIB reports, 'US' for the 456 NTSB reports - 1,252 documents in all.

What is actually populated

The schema documents the full system, but the database as delivered populates only part of it. Know which layers carry data before you build on them.

How the data was produced

The data flows through pipeline stages, and the tables mirror them. Provenance is preserved as a chain of links, not copied data: any finding traces to its passage, the passage to its sentences, each sentence to its document and position.

How the tables fit together

Two incident-data sources are kept apart and linked, not merged: the report PDF's own account lives in document_particulars / document_vessels (both corpora); the MAIB Data Portal spreadsheet's account in occurrences / vessels / affected_persons (MAIB only); the two join through a nullable documents.occurrence_id FK - several reports can point at one occurrence (e.g. an interim then a final). Tree-shaped classifications (event type, ship type, place on board, injury, deviation) are normalised into taxonomy_terms. The full entity-relationship diagram:

− 100% + Fit width Actual size Full screen Scroll to pan, or hold Ctrl/Cmd and scroll to zoom.

The data dictionary is authoritative for table and column definitions; the diagram is the map, the dictionary the legend.

Where each value comes from - and how far to trust it

This is the point of the dictionary. The database shows you structure; it cannot tell you which values are source fact, which a model produced, and which a human has since confirmed. Every column is labelled with one of the five below - the first two both mean "the source said so", and differ only in which source: a report PDF, or the MAIB spreadsheet.

A slashed label - Pipeline / AI-U - marks a column whose extractor is deterministic with a model fallback.

Two rules run throughout. Row-level verification: tables with an is_verified flag hold AI-generated content whose status varies per row - treat is_verified = true as AI-V, everything else AI-U. Sentence text is extracted by a vision LLM reading each page image, with deterministic passes on top, so it is AI-U, qualified by the parent documents.status (verified = analyst-confirmed).

Known limitations

Read these before drawing conclusions from the data.

• Two jurisdictions, not a global sample. Reports come from the UK MAIB (796 documents, published 2015-2026) and the US NTSB (456 documents, 2010-2026) - patterns describe UK and US waters and the vessels those two bodies investigate, not global shipping. Split or control by documents.jurisdiction: the corpora differ in body, report format, date span and field coverage, so pooling them silently mixes populations.
• Different reporting thresholds either side. Each body investigates only accidents meeting its own criteria, and those criteria differ. This is not a random sample; minor and unreported occurrences are absent by construction, so incidence rates cannot be estimated from it, and MAIB/NTSB counts are not comparable as rates.
• Spreadsheet incident data is MAIB-only, and sparsely linked. occurrences, vessels, affected_persons and affected_persons_ppe come from the MAIB Data Portal and cover UK incidents alone; no NTSB document has an occurrence_id. Even on the MAIB side the report↔occurrence link is partial: 58 of the 796 MAIB documents carry one. Treat the occurrence tables as an incident dataset in their own right, not as an enrichment of every report.
• Spreadsheet coverage window. The MAIB Data Portal extract feeding occurrences covers roughly 2018 onward - it is not a full historical record, and an older report may have no matching spreadsheet occurrence.
• Report-stated particulars are sparser for NTSB. document_particulars and document_vessels exist for every document, but fields are filled only where the report states them, and NTSB reports state fewer. Of 456 NTSB / 796 MAIB rows: accident_date 198 / 796, accident_location 208 / 731, severity 71 / 305, accident_type 130 / 279, loss_of_life 33 / 183; document_vessels.port_of_origin and destination are empty for NTSB (286 / 262 for MAIB), while vessel_name is better covered for NTSB (400 / 299).
• authors is not canonicalised. Its 22 rows carry PDF-metadata name variants of the two bodies (MAIB, Marine Accident Investigation Branch, www.maib.gov.uk; NTSB, National Transportation Safety Board, the typo NSTB) plus individual people named as a PDF's author. Use documents.jurisdiction to identify the publishing body, not authors.name.
• Text extraction is AI-driven and imperfect. Each page was rendered to an image and read by a vision LLM, with deterministic passes on top. Expect residual artefacts: merged/split sentences, headers absorbed into text, occasional transcription slips, and - where documents.extraction_fallback = true - a degraded result for some pages. 201 documents are flagged that way (182 MAIB, 19 NTSB).
• Sentence text is AI-produced, not byte-exact. It is the model's reading of the page image - a faithful-but-not-verbatim transcription. status = 'verified' documents are analyst-confirmed; extracted ones are not. As delivered all 1,252 documents sit at extracted - none has been analyst-verified. The source PDFs ship with the dataset, so anything that matters can be checked against the original page.
• Mixed categorical vocabularies. Report-stated fields (document_particulars.accident_type / severity, document_vessels.vessel_type) are free text - normalise before counting, and note MAIB and NTSB word them differently, so the free text is doubly heterogeneous. Identifiers differ too: publication_number is a MAIB Publication_No for UK reports and an NTSB report number (MAB-, MIR-, MAR-) for US ones, and document_type is a GOV.UK publication class that is null for every NTSB document. Spreadsheet classifications are normalised into taxonomy_terms and referenced by *_term_id. Coordinates are verbatim text, not validated geo types.

What is not included

• Vector embeddings - the schema has no embedding columns; nothing pgvector-dependent is present.
• Pipeline observability (LLM traces, extraction logs) and any application authentication data.

Row counts

As of 17 August 2026. Populated tables carry data; the analysis and findings layer is empty in the database (see above). Seeded vocabularies are loaded reference data.

Tandem Creative Dev · prepared for Zig Zag AI / CHIRP, for the Alan Turing Institute Data Study Group · 2026
Documentation conventions: data dictionary follows DAMA-DMBOK; this overview follows the Datasheets for Datasets framing.