Insight Engine ยท end product

What we build, and what OpenAI provides

The finished Insight Engine is a free public chatbot that answers questions from CHIRP's verified incident reports. This page shows the parts behind it: the LLM model we rent from OpenAI, the harness we build around it, the database it answers from, and how reports get into that database.

The one thing to take away

The OpenAI API is text in, text out. We send it one request and get one reply. It is stateless: it keeps nothing between requests, so it does not remember the conversation, search the web, draw charts or read our database. Everything that makes ChatGPT or Claude feel capable is a harness their makers built around the model, and for the Insight Engine we have to build that harness ourselves.

Simple diagram: you ask the chatbot; our harness remembers, reads uploads, searches CHIRP's reports and cites sources, fed by a qualitative layer (end user training, SHIELD coding, a maritime analysis schema and graph-based language processing); it sends words to the OpenAI model and gets words back; answers come from CHIRP's checked report database
Navy: built by us. Orange: OpenAI. Open full size

The four parts

Rented from OpenAI

The LLM model

Reads text and writes text, and is stateless. Each request stands alone: anything it needs to know, including what was said earlier in the conversation, has to be sent again every time. Its reply is either an answer or a request for us to fetch something.

Built by us

The harness

Turns a stateless model into a chatbot. It handles memory, uploads, database searches, legislation look-ups, usage limits and cost caps, and charts if they are brought into scope. It also checks that every claim in an answer cites a verified report.

Built by us

CHIRPdb

The database the chatbot answers from. Every report is broken into sentences, recommendations, safety issues, people, vessels and places, indexed by meaning, with the original PDF kept. The public only ever sees what CHIRP staff have verified.

Built by us

Behind the scenes

The ingestion pipeline reads new reports. The review tool lets CHIRP staff approve what was extracted. The feature sweep adds new detail to every report when users' questions call for it. The first two already exist; Tranche 2 points them at ~15 more jurisdictions.

One question, start to finish

Take a seafarer who types: "I'm about to enter a cargo tank. What's gone wrong for others doing this?"

  1. They ask. The harness checks usage limits and screens for misuse. Any attached photo or document is held for this conversation only.
  2. The harness picks out features. "Enclosed space", "cargo tank", "tanker". Features the database already records can be searched straight away.
  3. One request goes to OpenAI. It contains instructions, the conversation so far, the question, any attachment, the reports found so far, and a list of things the AI may ask us to fetch.
  4. One reply comes back. Usually the first reply is something like "search for enclosed-space incidents on tankers", not an answer.
  5. The harness fetches it. from CHIRPdb, adds the results to a fresh request and sends that. Steps 3 to 5 repeat until the reply is an answer.
  6. The harness checks the answer. Every claim must cite a verified report, or it isn't shown.
  7. The seafarer gets a reply in the chat: what has gone wrong in similar entries, with each point linked to the report it came from. They can ask a follow-up, and the conversation carries on.

One question can mean several OpenAI requests. Each one is billed, which is why the harness enforces a spending cap.

Getting faster with every question

Users will ask about things nobody thought to record. When a question depends on a feature the database doesn't hold yet, the harness starts a feature sweep. The sweep sends each report's text to the AI, one request per report, asks whether that feature applies, and stores the answer against the report.

Example. Someone asks how many groundings happened at night. "Time of day" isn't a recorded field, so the first answer takes a sweep across the corpus.

After the sweep, "at night" is a learned feature on every report. The next person to ask anything involving night-time gets an immediate answer from the database, with no sweep and far fewer AI calls.

Learned features are marked as AI-found until a reviewer confirms them, in keeping with the rule that AI proposes and humans verify. Answers that rely on them say so.

What comes with the API, and what we build

What users expect From OpenAI? What we build
Understanding a question, writing an answer Yes The instructions and the records it writes from
Remembering the conversation No Storing every turn and resending it with each request
Answers drawn from CHIRP's reports No The database, its search, and the fetch loop
Every claim cited to a source No Citation checks against verified records, and links to the PDF
Charts, tables, heat maps, timelines No Not yet in scope. If wanted, the harness would draw them from database figures
Reading a photo or uploaded document Partly The model can read what we send it; we handle upload, storage and deletion
Looking things up on the web No Legislation look-ups, flagged for the user to check
Getting faster and smarter with use No Learned features and the feature sweep
Safe for the public, affordable to run No Usage limits, spending cap, misuse checks, monitoring; room for paid tiers in Tranche 3