CHIRPdb review tool · User guide

The reviewer guide

The CHIRPdb review tool is a website where CHIRP staff check what the pipeline (the automated system that reads each accident investigation report) extracted from a report, correct anything it got wrong and sign the document off as verified. Your invite gives you the tool's web address and sets up the password you sign in with. Every button this guide names is named as it appears on screen, and the guide calls the website "the tool" throughout.

1Getting in

Your invite

Accounts start as an invite link from Tandem.

To use it:

  1. Follow the link. The page is headed Set Your Password.
  2. Choose a password of at least six characters and type it in both boxes.
  3. Press Set Password and Sign In. That signs you straight in; there is no separate confirmation step.

A link that answers with an error has been used already, has run out or has been replaced by a newer invite. Ask Tandem for a fresh one.

Signing in later

Go to the tool's address. The sign-in page asks for the email address your invite was issued to and your password. A refused sign-in says why in a red box: a wrong password, a deactivated account or an account that no longer exists.

You may see one of two messages here before typing anything:

On the sign-in page Your session expired. Sign in again. The tool renews your session quietly while you use it, but one left idle for long enough cannot be renewed. Nothing you saved is lost.
On the sign-in page Signed out. You pressed Sign Out, in the bar across the top of every page. A deactivated account gets its own message instead.

Press Sign Out when you leave a shared computer: it ends your session on the server as well as in the browser, so the back button will not bring a page back.

Forgotten password

There is no reset form. An administrator issues you a new invite for the same email address; redeeming it sets a new password and leaves everything else about your account as it was, unless the administrator changed your name, role or access when issuing it.

2What your role lets you do

Your role is shown beside your name in the bar across the top of every page.

RoleCan
readerRead the documents their account allows. No decisions, no edits, no sign-off, no history.
reviewerEverything a reader can, plus deciding, correcting and flagging every item and signing a document off.
analystEverything a reviewer can, plus run or re-run the pipeline (outside this tool).
adminEverything, plus returning a verified document to extracted. Accounts and invites are managed by Tandem for now; the tool has no screen for them.

A reader's page shows no decision buttons at all. The rest of this guide assumes the reviewer role or above.

3The queue

The queue is the tool's front page, reached from the Queue link in the bar across the top of every page. It lists the documents you may see, whether still waiting for review (extracted) or already signed off (verified), twenty to a page. Use Prev and Next (above and below the table) to move through them.

Columns

ColumnWhat it shows
TitleThe report's title; click it to open the document.
Statusextracted the pipeline's output, nobody has signed it off. verified a reviewer signed it off, with their name and the date beneath.
JurisdictionThe investigating body's country code (UK, US and so on).
ExtractionDegraded when reading at least one page fell back to a slower path; the text of that document deserves a closer look. A dash otherwise.
PublishedThe report's publication date.
Item StatesOne chip per review state the document's items are in, with a count: how much of it is still unreviewed, how much is flagged.
ClaimWho holds the document: whoever opened it while it was unclaimed or whoever last saved a change to it. You when it is yours.

Hover the small ? beside a column heading for the same explanation in the page.

Sorting

The Sort control sits top right: pick a column (Created, Publication Date, Status, Jurisdiction, Degraded Extraction, any of the five state counts or Claim Holder), pick Ascending or Descending, then press Sort. Changing a dropdown on its own does nothing until you press the button.

Filters

Filters opens a drawer on the right. It offers status, jurisdiction, extraction (degraded or not), "has an item that is" in a given state, claim (mine, someone else's or unclaimed) and a publication date range. Press Apply. The filters in force then appear as chips under the heading, each with an × to remove just that one, and a Clear all link. The Filters button shows a count of how many are in force.

Two useful filters: Claim set to Mine to find what you were in the middle of, and "has an item that is" Flagged to find what needs a second opinion.

4Claims

A claim is a note saying who is working on a document and since when. Opening an unclaimed document claims it. If someone else holds it, opening leaves their name in place; your first save moves the claim to you. The queue's Claim column and a band across the top of the document page show the holder's name and the time they opened it.

A claim does not lock the document and gives its holder no priority. Someone else can open the same document and edit it; the claim then shows their name instead. If two people change the same item, the second change is refused and the page shows what the first person did (section 8 covers this). A claim ends in one of three ways:

The band's colour says whose it is:

Orangeyours
Bluesomeone else's
Plainnobody holds it

5The document page

The top

The report's title, with two buttons beside it: Source, which opens the publishing body's own web page for the report in a new tab, and PDF, which opens the stored copy of the report in a new tab. If there is no stored copy the PDF button is greyed and reads No PDF stored; if the copy could not be reached just now, PDF unavailable.

Beneath the title, a strip of facts: status, jurisdiction, publication date, page count, report number and whether extraction was degraded. None is editable here; the jurisdiction, the publication date and the report number (as Publication Number) are corrected in the Metadata section below.

Beneath that, the review band: the claim on the left (section 4) and sign-off on the right (section 9).

The left rail

The rail is the column down the left of the document page, holding three panels: Sections, Pipeline and View. On a wide window it stays on screen as you scroll; on a narrow one (under about 1100 pixels) it moves above the document and scrolls with the page.

Sections lists the seven sections with each one's item count and a thin bar coloured by the states its items are in. The heading counts how many sections are open and how many have unreviewed items. Clicking a section name scrolls to it and unfolds it if it was folded.

Pipeline shows the six stages the document went through, each with a dot and the date it finished. A red dot is a failed stage, with the reason beneath; a hollow one is a stage that did not run. A document whose records stage failed will have an empty Records section, and this panel is where that is explained.

View has one link: Hide or Show rejected items. Rejected items are shown by default to anyone who can review, greyed, so you can see what was thrown out and reverse it if need be. Hiding them takes them off the page; the two fact panels then say how many fields are hidden, and the other sections leave them out.

The seven sections

Each section is a panel with a title band. Three of the bands fold the panel away: Records opens unfolded, Entities and Addressees start folded. Inside Records, the Safety Issues and Recommendations sub-lists start closed; click either band to open it.

SectionHoldsReviewable?
MetadataFive document fields: title, publication date, publication number, document type, jurisdiction.Yes, each field
Accident particularsEleven facts about the accident: date, location, severity, accident type, loss of life, injury count, whether the vessel sank, total loss, pollution, property damage, severe environmental damage. Plus the vessels involved, read-only.Yes, each field
StructureThe report's own headings in order, each with a PDF link to its page. For finding your way around the report.No
Incident linkThe accident in the investigators' own data that this report was matched to, by its publication number.Yes, one item
RecordsThe report's safety issues and its recommendations, each in its own sub-list.Yes, each record
EntitiesThe vessels, people, places and organisations the narrative names, with every place each one is mentioned.Yes, each entity
AddresseesFor each recommendation, the organisations it is directed at.Yes, each recommendation's list

If one section cannot be loaded (the request was refused or timed out), that section says so and the others still render. Reload the page; if it persists, tell Tandem.

6States and decisions

The five states

Every reviewable item is in one state, shown as a coloured chip on the item and as a coloured stripe down its left edge.

StateMeaning
unreviewedNobody has looked at it. The pipeline's value stands, unconfirmed.
acceptedA reviewer confirmed the pipeline's value as it is.
correctedA reviewer changed it. The value shown is the reviewer's; the pipeline's original is in the history.
rejectedA reviewer said this item should not exist (a recommendation that is not one, an entity that is noise). It drops out of what readers of CHIRPdb see.
flaggedA reviewer is unsure and wrote a note saying why. A flagged item stops the document being signed off until someone resolves it.

An item can be decided any number of times. The latest decision is its state; every earlier one is in its history.

The toolbar

Each item has the same toolbar on its right (beneath it on a narrow window). On the fields in Metadata and Accident particulars and on entity cards, where room is short, Accept and Reject are a ✓ and an ✕; hover for the word.

A reader's view has none of these.

Last change and History

Beneath an item's chip, a grey line names who last changed it and when ("Jaz accepted, 14:30"; a change from another day carries its weekday). Beneath the item's text, ▸ History unfolds the item's whole trail, newest first: each decision or edit, who made it and when, its reason if one was given; an edit also lists each field's value before and after. At the bottom, Machine original: the values the pipeline produced before anyone touched them. An item a person created from nothing says so instead.

History is read fresh each time it is opened, so it includes what someone else did a moment ago.

7Correcting each kind of item

Metadata fields and accident particulars

Each field shows its value and, in a small chip, where that value came from: landing page, PDF properties, cover page, report text, model reading, derived from other fields or, once a reviewer has changed it, reviewer edit. A field with no chip had no recorded source.

Edit opens the field in place. What it offers depends on the field:

FieldEditor
Title, publication number, accident location, pollutionA text box.
Publication date, accident dateA date picker.
Loss of life, injury countA whole number, zero or more.
Property damageAn amount, zero or more. The label names the currency the jurisdiction reports in.
Vessel sank, total loss, severe environmental damageYes, No or Unknown.
Document type, jurisdiction, severity, accident typeA list of allowed values. The list is read when the editor opens and depends on the document's jurisdiction; a short "Loading the list…" is normal.

Title and jurisdiction cannot be emptied. Every other field can, by clearing the box; "(none)" at the top of a list does the same.

Save is disabled until the value differs from what is stored. A saved edit moves the field to corrected and records it as a reviewer edit. Editing the title, the publication date or the jurisdiction reloads the whole page; a changed jurisdiction also changes what the other lists offer.

Two notes you may see on a field. "Read as US, not this document's current jurisdiction" means the pipeline read this value under a different body's conventions than the document now carries, usually because someone corrected the jurisdiction since; a re-run of the pipeline would read it afresh. The jurisdiction editor itself carries the matching warning. Neither note needs action from you beyond checking the value.

Records: safety issues and recommendations

A record is one safety issue or one recommendation the report makes, kept as the sentences it spans. The Records section has two folding sub-lists, Safety Issues and Recommendations, each with its count and state chips in the band. Above a list of more than one, Show narrows to one state and Sort orders by report order, review state (undecided first) or wording; the arrow button reverses the order.

A record's words are the sentences of the report it spans, shown as its text, with a PDF pill that opens the report at that passage's page. A safety issue shows whether it was contributory and whether it was addressed; a recommendation shows its reference number and the addressee named in the report.

Edit Fields opens the record's own fields. For a safety issue: Name, Kind of record (Safety issue or Safety lesson), Contributory and Addressed (Yes, No or Unknown). For a recommendation: Reference number, Addressee and Action. Only the fields you changed are sent; Save Changes stays disabled until something differs.

Edit Span changes which sentences the record covers. It lists the sentences of the record's own section, with the ones it spans ticked. Tick and untick as needed; Show the previous section and Show the next section extend the list when a record runs across a heading. A record must span at least one sentence. Save Span saves the new span and moves the record to corrected.

Entities

An entity is a vessel, person, place or organisation the report's narrative names, with every place it is mentioned. The section is paged, fifty cards at a time. The controls above the cards filter by state and by type, sort (by doubt, name, mentions or review state) and need Apply pressed; Show all drops the filter. The default order puts the most doubtful first. An entity the pipeline was unsure of carries a small amber note naming the doubt:

NoteDoubt
otherIt fitted no known type.
low_confidenceAt least one mention scored low or was not scored.
name_mismatchA mention's wording disagrees with the name.

The note hides once you have changed the entity but returns when the list reloads: it describes what the pipeline produced, whatever you decided.

Each card shows the entity's name, its type, how many times it is mentioned and its state. Besides the standard toolbar:

Two buttons above the cards accept in bulk. Accept All on This Page accepts every unreviewed entity on screen. Accept All on the Document accepts every unreviewed entity on every page, whatever the filter shows; when that is more than the page in front of you it asks first and tells you how many. Both take an optional reason. An entity someone else changed while you looked is skipped and named in the notice, so nothing of theirs is overwritten.

The investigators' own data (for the UK, the MAIB data portal) lists every accident; the pipeline matches a report to its accident by the publication number. This section shows the match, or No linked incident.

The buttons read differently here:

Addressees

An addressee list is the set of organisations a recommendation is directed at. The section has one row per recommendation: its reference and the addressee as the report worded it, then the organisations as chips. + Organisation opens a search box over the organisations the database knows; pick one to attach it. The × on a chip detaches that organisation. Creating a new organisation is not possible here; if the right one does not exist, flag the row with its name in the note.

Each row's Accept List and Reject List decide the list as a whole. Attaching or detaching moves it to corrected on its own.

8When a change is refused

Your change is refused when someone else changed the same item after you opened it. The tool sends, with every change, the moment you last read the item; if that no longer matches, a yellow message appears on the item and the item redraws showing what is stored now, including their decision. Read it, then decide again if you still need to. Nothing of yours was saved by the refused change, so there is nothing to undo.

A different message Something went wrong saving that. Reload the page and try again.

That one means the tool lost its connection. Reloading usually clears it. If it does not, tell Tandem.

9Sign-off

A document starts as extracted. Signing it off makes it verified, which is the attestation CHIRP's readers rely on: a reader whose account is limited to verified documents sees it from that moment.

The band at the top of the page reads Not yet signed off, with a Sign Off button for anyone who can review. The confirmation says what will be recorded: your name and the count of items in each state, section by section (the time is recorded too).

Two conditions, both checked at the moment you press the button:

Condition 1
Nothing on the document is flagged.
Condition 2
Every document-level item is decided: the five metadata fields, the eleven accident particulars and the incident link.

Records, entities and addressee lists may stay unreviewed; the sign-off records how many were. A refused sign-off shows a yellow box listing each item in the way, with its state, each one a link that scrolls to the item and unfolds its section. Resolve them and press Sign Off again.

Once signed off, the band reads Signed off by with the name and date, and What it covered unfolds the recorded counts. The button becomes Sign Off Again, for when further corrections have been made since: the new sign-off replaces the old one's name, time and counts.

Returning a document

Only an administrator sees Return to Extracted. It takes the document out of view for every reader limited to verified documents at once and needs a note saying why; the note shows in the band, with the administrator's name, until the document is signed off again. The item decisions are untouched by a return.

10Reading the PDF

The report's own PDF is stored with the document. Every item that comes from a sentence carries a small orange PDF pill: each heading in Structure, each record, each entity mention. Clicking one opens the PDF in a new tab at that sentence's page. The PDF button beside the title opens it at page one. Fields in Metadata and Accident particulars carry no pill; their source chip (section 7) says where the value came from.

The link to the PDF is created when you click and lasts an hour; a tab left open longer than that shows an expired-link error when reloaded. Click the pill again.

11A working day

  1. Open the queue, filter by Claim: Mine for anything you left mid-way; otherwise sort by Unreviewed Count descending and open the top document.
  2. Read the Pipeline panel for a failed stage, and the strip beneath the title for Degraded extraction. Either is a reason to read that document's text more carefully against the PDF.
  3. Work the two fact panels first (Metadata, Accident particulars): every field needs a decision before sign-off.
  4. Confirm or clear the incident link, or search for the right accident and link it.
  5. Work Records: Edit Span where the pipeline cut a recommendation short, Edit Fields where it misread a reference number, Add a record it missed altogether, Flag where you cannot tell. Sort by review state to keep the undecided at the top.
  6. Entities: sort by doubt, decide the doubtful ones by hand, then Accept All on the Document for the rest if a scan of the names finds nothing wrong.
  7. Addressees: attach the organisations each recommendation names, then Accept List.
  8. Sign Off. If it is refused, follow the links in the yellow box.

Half a document is fine to leave. Your claim and every decision are saved as you go; there is no draft to lose.