Handle re-runs, new versions and failures

Measure the classifier and correct a wrong answer

By the end of this lesson, you can classify a document again and tell which results the classifier keeps and which it replaces.

Lesson 3 of 5~25 minScript: 03_rerun.sql, 03_new_version.sql, 03_failures.sql

Classify a document again

This lesson continues with the files and verdicts of lesson 2. A document is classified more than once: after you change a description, after a new version, or when a person asks. The procedure hkd_ai_api.reclassify queues the event AI_RECLASSIFY for a document. Only the AI workflow reacts to that event, so the other workflows of the hooks demo do not run again:

hkd_hook_api.enqueue_event('AI_RECLASSIFY', p_document_id => p_document_id, p_version_id => l_version_id);

A second run can meet results that an earlier run or a person left behind. The classifier replaces its own earlier verdict. It keeps what a person decided. The next section covers a category tag that a person set, which has two cases. Lesson 4 covers two related decisions: a verdict that a person accepted, and a release that a person took back.

Keep a category tag that a person set

The classifier runs as the system user. A tag that another user wrote was set by a person, so the classifier does not overwrite it. The tag also changes the release:

If the model agrees with the tag of the person and the type releases automatically, the verdict is released. The tag stays as it is, and the summary of the run says that the classifier kept it.
If the model disagrees with the tag, nothing is released. The verdict waits for review and the tag of the person stays.

category_set_by_person(p_document_id, l_person, l_person_type);

if l_valid and l_mode = 'AUTO'
   and (l_person is null or l_person_type = l_doc_type) then
  l_status := 'RELEASED';
else
  l_status := 'NEEDS_REVIEW';
end if;

Run @03_rerun.sql. The user ADMIN sets the category of the datasheet to other, and then the document is queued again. The model still answers datasheet, so the case is the second one. The first case works in the same way and the script does not run it:

The queue is empty (waited 3 s).
WORKFLOW_CODE    STATUS     SUMMARY
________________ __________ ______________________________________________________________________________________________________________________
AI_RECLASSIFY    FLAGGED    Classified as datasheet, waiting for review (383+33 tokens). The category tag "other" was set by ADMIN and was kept
TAG_NAME    TAG_VALUE    UPDATED_BY
___________ ____________ _____________
category    other        ADMIN

The status is FLAGGED: the verdict datasheet waits for review, the tag other stays, and the summary says who set it.

A queue message is only a pointer to a document and a version. If a person deleted the document before the worker runs, the classifier ends the run with SKIPPED and makes no model call.

Classify a new version of a file

A new version can be another kind of document, so the version hook queues an event as well. The workflow AI_CLASSIFY_VERSION classifies the new version with the same code. Run @03_new_version.sql. It saves an amended copy of the supply agreement as version 2:

l_version_id := adm_document_api.add_document_version(
  p_document_id  => l_doc_id
, p_file_content => apex_web_service.clobbase642blob(l_b64)
, p_allow_merge  => false
);

p_allow_merge => false always adds a version. With the default and a merge window above 0, a save shortly after the previous one replaces that version.

Saved version 12477395917275516799091732396492 of document 5177945473410070129386129567827
The queue is empty (waited 3 s).
WORKFLOW_CODE          STATUS     SUMMARY
______________________ __________ _____________________________________________________________
VERSION_DIGEST         OK         Version 2: size 2.3 KB -> 2.3 KB (+1%). Notified ADMIN
AI_CLASSIFY_VERSION    FLAGGED    Classified as contract, waiting for review (476+32 tokens)
   VERSION_NUMBER DOC_TYPE    VERDICT_STATUS    CREATED_AT
_________________ ___________ _________________ ______________________________
                1 contract    NEEDS_REVIEW      02.10.26 12:00:22,160162000
                2 contract    NEEDS_REVIEW      02.10.26 12:02:09,541698000

The version hook ran two workflows: VERSION_DIGEST from the hooks demo and AI_CLASSIFY_VERSION. The new version has its own verdict. The type contract needs review, so the verdict waits for a person. The verdict of version 1 stays in the history.

Handle a file the model cannot read

Some errors repeat on every attempt. The script 03_failures.sql uploads a file that has the name and type of a PDF but no PDF inside. The provider refuses it with HTTP 400:

The queue is empty (waited 3 s).
WORKFLOW_CODE         STATUS       ATTEMPT SUMMARY
_____________________ _________ __________ __________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
DUPLICATE_SCAN        OK                 1 Unique content, 23 B
AI_CLASSIFY_UPLOAD    FAILED             1 The model call failed and will not be retried: ORA-20302: Error response from provider Responses API: HTTP 400 from provider, response: {   "error": {     "message": "The file you uploaded is badly formatted or corrupted. Please fix the file and t
STATUS    REASON
_________ ___________________________________________________________________________________________________________________________________________________________________________________________________________
FAILED    ORA-20302: Error response from provider Responses API: HTTP 400 from provider, response: {   "error": {     "message": "The file you uploaded is badly formatted or corrupted. Please fix the file and t

The classifier records the failure and does not raise an error. The queue does not deliver the event again. The workflow has one run, attempt 1, with the status FAILED. The table hkd_ai_results keeps the text of the error. A document with a FAILED verdict has no verdict that a person can review. The functions enqueue_unclassified and enqueue_outdated of lesson 5 do not queue it again. After you fix the cause, call hkd_ai_api.reclassify for the document.

The classifier separates two kinds of failure:

Kind	Examples	What the classifier does
Permanent	The provider refuses the file (HTTP 400), or a request is wrong	Records `FAILED` and returns. A new attempt would fail in the same way.
Transient	The provider is busy (HTTP 408, 425, 429 or 5xx), or the network connection fails	Raises the error. The worker rolls the event back and the queue delivers it again.

A transient failure has a price. The queue delivers the event up to three more times, and each attempt sends the file again and uses tokens again. The hooks course shows the retries and the exception queue in Handle a workflow that fails. This lesson does not simulate a transient failure.

Review and release verdictsAccept, change or take back the verdicts that wait for a person, and check a sample of the released ones.