We won't quote a percentage
until we've seen your orders.
A lot of vendors quote the same kind of figure: 87% automatic, from twelve minutes down to 45 seconds. Nobody says where those twelve minutes come from. There is also no public measurement of how long order entry takes in Dutch wholesale, not from Statistics Netherlands and not from the trade associations.
We care most about one number: how many orders go through wrong without anyone noticing. Hardly anyone publishes it. We put it first.
| Measure | Clean | Typos | Scan noise | Synonyms | All at once |
|---|---|---|---|---|---|
| Order lines correct | 100.0% | 91.3% | 94.0% | 97.8% | 85.2% |
| Orders with no manual work | 54.5% | 34.8% | 34.8% | 50.0% | 22.7% |
| Wrong and let through unseen | 0 | 0 | 0 | 0 | 0 |
This is our own test set: a made-up wholesaler with 22 documents and 61 order lines. The documents were read with fixed parsing rules we wrote for this set, not yet with the language model. So this is not a customer result. For the damaged columns we damaged the same documents on purpose and took the average of three runs. The ‘Clean’ column is a single run on exactly the documents the rules were written for, so it says little.
Under the heaviest damage, 85.2% of lines stay correct, but only 22.7% of orders can go through without a person. That's on purpose: the less sure the system is, the more careful it gets. We measured the threshold above which a line may go through automatically on this test set, in one run. Below 0.90 things sometimes went wrong; above it they didn't. Whether that holds for the language model and your orders is what the baseline measurement shows. Automatic approval starts switched off: you switch it on yourself, once the baseline measurement on your orders supports it.