Form extraction manager

The form extraction manager is an APEX region plug-in, <adm-form-extraction-manager>, in the AI Pack. Use it to configure a form extraction profile, test forms on real documents, review and approve results, keep a test set and measure the quality of the configuration. You do not have to write configuration JSON or call the PL/SQL API.

The plug-in shows one profile. The list of profiles is an ordinary APEX Interactive Report on its own page. Select a profile in the report to open the page that holds the plug-in.

Only administrators can use it. Every method checks this on the server. A user who is not an administrator gets a no-permission error, and the region shows a short message instead of the manager.

Before you start

• Install the AI Pack. See Installing the AI Pack.
• Read The three levels of the configuration. The manager uses the same profiles, document types and forms.
• Ask a developer to add the plug-in to your APEX application. The steps are in Add the plug-in to an APEX page.

Create a profile

1. In the profile report, select Create. The page opens the form New profile. To change a profile later, open it and select Edit. The form then has the title Edit and the name of the profile.
2. Read the short introduction at the top of the form. Below it, the disclosure How ADM reads a document is open on a new profile and closed when you edit. It holds the table that is explained in How ADM reads a document.
3. Fill in the sections in this order:
   • Name: Profile name is required. Use lower case letters, digits, underscore and hyphen. Description is optional.
   • Which documents: fill in at least one of Folder and Documents with an annotation. If you fill in both, a document counts when either one matches. Folder selects the documents in the folder and in the folders below it. Select Browse… to pick a folder. Documents with an annotation takes an annotation key. It selects the documents that have the annotation, and the documents in a folder that has it. A line under the fields says what your entries select.
   • How documents are read: one to three readers. Only the text model is required.
     • Text model: Provider, Model and APEX web credential. It reads the values of every document that has text.
     • Use a vision model for images: optional. If you select it, the same three fields appear for the vision model.
     • Use OCR for scans: optional. If you select it, enter the OCR service and, for Ollama, the OCR model. File types, Only documents with this annotation and Annotation value limit the documents that OCR reads.
   • Instructions for the model: Rules are short instructions that go into every request to the model. Select Add rule to add one. Rules for one form or one document type go on the Document types & forms tab.
   • When a person must check: Review tag. Under Advanced settings: Minimum text and Maximum text.
   • When to read: Read new documents automatically. Under Advanced settings: Documents in one run.
4. Select Create profile. When you edit a profile, select Save.

APEX web credential is the static ID of an APEX web credential. Do not enter the API key itself: put the key in the credential. Leave the field empty if the provider needs no key. Each model has its own Advanced settings: Server address, Reasoning and Reasoning level (text model and vision model only), and Other provider settings (JSON). The box opens by itself when a field in it has a value or a message.

The form does not fill in defaults. Review tag, Minimum text and Documents in one run show the default as a grey placeholder. If you leave a field empty, ADM does not write the key, and the server saves the default. Maximum text has no default: if you leave it empty, there is no limit.

Check the form before you save

The form checks its own rules while you type, and the server checks the configuration after each change. A field shows its messages after you leave it, or after the first attempt to save. The Create profile button is off until the server has checked the configuration. When the configuration is valid, the line The configuration is valid. shows next to the buttons.

If you select the button while a rule is broken, the form shows the list N problems to fix and moves the focus to it. Each line names a field, for example Which documents, and links to it. A message of the server that belongs to no field, for example an unknown key, shows in the same list.

validate_profile returns the broken rule of every section of the configuration, not only the first one found. Each message has the path of its field. A section reports one broken rule at a time, so a section can show a second message after you correct the first. A rule that needs a value from a broken section is not checked until you correct that section.

Edit the configuration as JSON

The Advanced (JSON) tab shows the stored configuration with its key names. Edit it there, and select Apply to form to load your text back into the form. Until you apply or discard your changes, you cannot save. A key that the form has no field for stays in the configuration when you save.

How ADM reads a document

ADM picks the reader from the type of the file, and from how much text the file has. The table is the one on the form.

The document is	ADM does this	It needs
A PDF or other file with text	Takes the text, finds the form with its markers, and the text model reads the values.	Text model
A scan (a PDF with too little text)	OCR reads the pages. Then the text model reads the values.	OCR and text model. Without OCR, the document goes to review.
An image (PNG, JPEG, and more)	The vision model looks at the image, finds the form and reads the values.	Vision model. Without it, the image goes to review. If OCR covers the image type, OCR and the text model read it instead.

These rules apply to every document:

• OCR reads a document only if all of these are true: the profile has an OCR section, the setting AI_TEXT_EXTRACTION is Y, and the file type is in File types. If you set Only documents with this annotation, the document must also have that annotation. If File types is empty, the server saves application/pdf. To read images with OCR, add image types such as image/png, image/jpeg.
• OCR is its own service. It is not a model of the text provider. The setting AI_TEXT_EXTRACTION is Y by default. See Settings. If it is not Y, OCR reads no document, and the manager shows a warning below Use OCR for scans. The form gets this value from get_config_options (text_extraction_enabled).
• An image goes to the vision model, unless OCR reads it. If an image type is listed under OCR, OCR wins, and the vision model does not see that image type. The vision model still reads all other images.
• If an image has no reader, it goes to review and ADM calls no model. This is the case without a vision model, and also when the profile has OCR but not for this image type.
• A scanned PDF never goes to the vision model, because only files of an image type do. A scan that is saved as an image file is an image.
• ADM reads every other file as text. It uses OCR text that ADM stored earlier, if that text has at least Minimum text characters. Otherwise it uses the text of the built-in text extraction. Blanks do not count.
• If the text has fewer characters than Minimum text and OCR reads the file, OCR reads it. If OCR does not read the file, the document goes to review and ADM calls no model.
• The markers of the forms choose the form, not the model. If no form matches, or more than one matches, the document goes to review. If no form matches, the text model can add a hint for the person who reviews it.
• If a form matches and the text has more characters than Maximum text, the document goes to review. The limit counts all characters, blanks included.
• The profile needs a text model, also if it reads only images. The server rejects a profile without one.
• ADM checks every result. If every check passes, ADM sets the annotations and removes the review tag. If not, it adds the review tag.

An administrator can also set a global OCR fallback with the setting AI_OCR_FALLBACK_CONFIG. It reads scans for all documents and stores the text. A profile then needs its own OCR only for a different service. See Settings.

Define a document type

A document type defines the fields that every form of the type returns, and the checks on them.

1. Open the Document types & forms tab, and select Create document type.
2. Enter the code of the document type, for example invoice. The annotations of an accepted result are named after it, for example invoice.total_amount.
3. In Fields of the document, add one row for each field: the name, the type (string, number, integer or boolean), Required, Allowed values and Annotation. Allowed values is an optional list of the values that a string field may have, separated by commas, for example invoice, credit_note. Annotation marks a field that becomes an annotation of an accepted result.
4. Select The document has line items when the form has a table of lines, and add one row for each column.
5. Add the Rules, the Checks and the Line checks.
6. Select Save.

The server checks each expression while you type. A valid expression shows the SQL that runs. An expression that is not valid shows the error, for example a name that is not a field. Select Which functions and names can I use? to see the allowed functions, keywords and aggregates. The rules are in Write check expressions.

A field that uses a part of JSON schema that the table cannot show, for example a format or a list of allowed values that has a comma in a value, is shown as Custom schema, edit it in JSON. Change it on the Advanced (JSON) tab. The form keeps it unchanged when you save.

You cannot delete a document type that a form uses. Delete the forms or move them to another document type first.

Define a form

A form is one layout of one issuer, for example the invoice of one supplier.

1. Open the Document types & forms tab, and select Create form. You need a document type first.
2. Fill in the sections:
   • Identity: the form code (capital letters, digits and underscore), the document type, the description and Active. The description is part of the prompt. Describe the document, not the layout.
   • Markers: regular expressions. All of them must match the text of a document.
   • Field patterns: a regular expression for a field. For a column of the line items, select line_items.<column>.
   • Line items: optional. A line pattern and the group of each column read the line items without the model. Select Decimal comma when the form prints 1.234,56.
   • Rules: rules for this form only.
3. Select Save.

Test a form on a document

Use Test the form on a document before you save, to make sure that the markers and the line pattern work:

1. Select Choose a document, and select Choose next to a document of the profile.
2. Read the result:
   • each marker, and whether it matches the text
   • the saved forms that match the document
   • the line items that the line pattern reads
   • the text that the markers get, and the raw text

The test uses the form as it is in the editor, including changes that you did not save. It calls no model and no OCR, and it writes nothing. It reads the same text as the extraction: OCR text that ADM stored earlier, or else the text of the built-in text extraction. The line pattern is not used on OCR text, as in the extraction.

Review the results

The Results tab lists the latest result of each document. Filter by status, or search by the document name. Select a document to open its result.

The result view shows the status, the method, the source of the text, the form, the model, the tokens and the time, and the issues. Select Show the text to see the text that the extraction read.

To correct and approve a result:

1. Select the form, if the extraction found none or the wrong one. The fields change to the fields of that form.
2. Correct the fields and the line items. Add line and Remove change the lines.
3. Keep Add to the test set selected to copy the approved values into the test set.
4. Select Approve.

The checks run again on the corrected values. When a check fails, the view shows the issues and changes nothing. When a hook refuses the result, the view shows the message of the hook.

When the result is from an older version of the document, Approve is off. Select Extract again to read the current version. The view sends the version that it shows, so a result that changed in the meantime is not approved.

Extract again extracts the document again and writes a new result. It waits for the model, which can take a minute. Save as test case writes the values in the form to the test set, without an approval.

Try a document

Select Try a document in the profile header, and select a document. The manager extracts it and shows the result. It writes no result and no annotation. Only the OCR text of the profile is stored, as for every extraction.

Set another model or the naive prompt under the options to compare a configuration before you change it. Then select Save as test case to keep the values for the test set.

Extract with saved settings runs a new extraction with the saved settings of the profile and writes that result. It does not use the options, and the new result can differ from the preview. The message that follows shows the status of the new result: published, needs review, or failed.

Keep a test set

The Test set tab lists the test cases of the profile: the documents with the values that a correct extraction returns.

• Select Add document to write a test case for a document. Select the form and enter the correct values.
• Select the form No form (must go to review) for a document that is no form. Its extraction must go to review.
• Select a test case to change it, or Delete to delete it.

An approval with Add to the test set also writes a test case.

Measure a configuration

A measurement runs the extraction on every test case and compares the result with the correct values. The outcomes are described in Measure a configuration.

1. Open the Measurements tab, and select Run measurement.
2. Enter a label. It must be new for this profile, because the label identifies the run.
3. Optionally limit the run to one form, or to the documents whose name contains a text. Optionally set another text model or vision model, the naive prompt, or the reasoning switch.
4. Select Start.

The measurement runs in a background job with your ADM context. It commits after each document, so the run shows in the list with the documents that it has measured so far. The page refreshes while the job runs.

The list shows the outcomes of each run. A count of MISSED above zero is highlighted: these are wrong values that would be published. Select a run to see each document with its expected and found form, the wrong fields and the issues. Select a document to open its result. Delete run deletes the rows of the run.

Process new documents now

Process now in the profile header starts the automatic processing for this profile at once, in a background job with your ADM context. It extracts the same documents as the AI queue job, and commits after each document. See Process new documents automatically.

The button is off when the profile does not process new documents automatically, or when a job of the profile runs. A processing run and a measurement of the same profile cannot run at the same time. A refused start is the error adm_ai_error.e_form_job_refused.

Add the plug-in to an APEX page

Do these steps in the APEX Builder:

1. Make sure that the region plug-in ADM_FORM_EXTRACTION_MANAGER is in your application. If you build your own application, create it with these settings:

   Setting	Value
   Type	Region
   Render procedure	adm_ai_form_plugin.render_region
   AJAX callback	adm_ai_form_plugin.ajax_region
   JavaScript file URL	#PLUGIN_FILES#adm-form-extraction-manager.min.js
   File	upload adm-form-extraction-manager.min.js
   Custom attribute 1	Scope Component, type Page Item, label Profile ID Item, required

2. Create the list page. Use an Interactive Report on the view adm_ai_form_profile_status_v. Link a column to the profile page and pass the profile id to the page item of that page. Add a Create button that links to the same page and clears its cache.

3. Create the profile page. Add a hidden page item for the profile id, for example P13210_PROFILE_ID. Add one region of the plug-in type and select this item in the Profile ID Item attribute.

4. Give both pages the authorization scheme @administration-rights, and the plug-in region and the report the scheme @ai-pack.

The region reads the page item when the page renders. If the item is empty, the region shows the form for a new profile. After you save it, the region shows the new profile and sets the page item to its id. This also holds after you reload the page.

Use one region of this plug-in on a page.

The view adm_ai_form_profile_status_v has one row for each profile: the source, the text model, whether new documents are processed automatically, the counts of document types, forms, results by status and test cases, and the label and date of the last measurement.

Also refer to

• Form extraction
• adm_ai_form_admin_api: the PL/SQL API behind the manager