Can AI track contract deadlines and fill in forms? We tested it on 114 documents.
The company in this test, Kroonmeer Installatie, is made up, and so is every name, number and document. Real work runs on your own data, and you check everything before it counts.
On these documents, yes. Our AI found 105 of 105 deadlines with the exact date in a made-up company's contracts, policies and letters, including dates moved by a later email. It filled 93 of 93 form fields right, invented nothing and signed nothing. It took 12 minutes. It also asked far more questions than it needed to.
What it got
- 114 documents: contracts, insurance policies, lease papers, permits, certificates, letters and emails, including 2 image-only scans
- 4 blank forms with 93 fields
- A note from the owner
Build a deadline register for the next 12 months, a briefing for the next two weeks, fill in the forms ready to sign, and list questions for the owner. The job came from an admin assistant's role at a 16-person heat pump installer.
The results
| Measure | Result |
|---|---|
| Deadlines found, right document and exact date | 105 of 105 |
| Briefing items for the next two weeks | 12 of 12 |
| Form fields filled right | 93 of 93 |
| Form fields made up | 0 |
| Signatures or signing dates filled in | 0 |
| Questions that had to be asked, found | 7 of 7 |
| Moved, cancelled or bogus dates still used | 0 |
| Past dates put in the future register | 0 |
A script scored every line against an answer key the AI never saw. How the tests work.
What it missed
- No required deadline was missed. One was listed twice.
- It asked 32 questions where 7 were needed. None of them were on form fields the documents could answer, but the owner would still have 25 extra questions to read.
- Two extra rows we hadn't listed. We checked both by hand and they were sound: they stayed in the register, and they aren't errors.
Problems we hid in it
These are ordinary things that cost money or trust when they slip through. This is a selection.
| What we hid | What it did |
|---|---|
| A deadline brought forward by a later email | New date used |
| An audit moved by email, with a "four weeks before" date that moves with it | Both dates right |
| A notice date you have to calculate from a lease's end date | Calculated right |
| A US-format date: 01/12/2027 meaning 12 January | Read right |
| A deadline on a Sunday that the document moves to Monday | Monday used |
| An official-looking "last reminder" that the small print says is an offer | No deadline added, flagged |
| Two documents that disagree on a lease's end date | Question asked |
| Last year's policy next to this year's | New policy number used |
| A date already passed | Kept out of the register, question asked |
What this test doesn't show
It read files, not your filing system. A real business also has deadlines that were never written down. The company's rule for who owns each deadline was given to it, so that score shows it can follow a rule, not that it can work out who is responsible. This was also one run, on one made-up company, and we wrote the test ourselves.
Would this work on your admin?
Send us the job ad you're hiring for, and we'll send back a free one-page breakdown of what we'd take off the role.