- Pilot
- Document analysis
A report from 120 documents: analyst work fell from eight hours to two
A pilot compared two ways of preparing a four-page information report from 120 documents. Human work took eight hours in the manual version and two hours with AI. The second figure includes preparing the data, checking it and making corrections. The tool worked for an additional 20 minutes.
Pilot. The solution stayed at pilot stage and was not used for further reports.
Problem
A four-page information report from 120 documents required 480 minutes of human work: 90 minutes of preparation, 300 minutes of review and drafting, and 90 minutes of checking.
The test
In the AI-assisted version a person prepared the material, checked the result and made corrections. The tool worked for an extra 20 minutes, and the scope and the list of 40 required items stayed the same.
Effect
Human work came to 120 minutes. The first version contained 38 correct items out of 40; after corrections the report met every requirement, exactly as the manual version did.
- Pilot result
- Human effort lower by six hours, that is 75%. This is the result of one comparison run on the same set of material.
Measurement
| Metric | Before | After | Change |
|---|---|---|---|
| Preparing the material | 90 min | 35 min | |
| Review and drafting | 300 min | 20 min of tool time | |
| Checking and corrections | 90 min | 85 min | |
| Total human work | 480 min | 120 min | −75% |
Basis of measurement: One comparison run on the same closed set of 120 documents and the same list of 40 required items. The reports were prepared by two people of similar experience, so the second run could not draw on the memory of the first.
Scope of the test
The scope was limited to collecting and organising information about a company. The report contained no financial assessment, recommendations or decisions. Both versions used the same closed set of documents and the same list of 40 required items.
Time measurement
The manual version covered 90 minutes of preparing material, 300 minutes of review and drafting, and 90 minutes of checking. In total that meant 480 minutes of human work.
The AI-assisted version covered 35 minutes of preparing material, 70 minutes of checking the result and 15 minutes of corrections. The person worked for 120 minutes in total. The tool ran for an additional 20 minutes. In this scenario the steps happened one after another without breaks, so the whole run took 140 minutes.
Human effort fell by six hours, that is 75%. This is the result of one comparison run. It refers to the same set of material, with the reports prepared by two people of similar experience so that the second run could not draw on the memory of the first. Differences between individuals remain a limitation of the comparison.
Quality control
A reviewer checked all 40 required items in each report. The first version prepared with AI contained 38 correct items and two that needed completing. After corrections the report met all 40 conditions. The manual report also met every requirement after checking. Checking and correction time was included in both versions.
What comes next
The owner received a basis for the next decision: repeat the test on a larger number of reports and check how it performs on documents of varying quality. The pilot was not yet a regular deployment.
- Conclusion
- The owner received a basis for the next decision: repeat the test on more reports and check how it performs on documents of varying quality.