Skip to main content

Bring Your Own LLM

DocuDevs can process documents using your organization's Azure OpenAI, OpenAI, or OpenAI-compatible account. Azure OpenAI and OpenAI are fully supported. OpenAI-compatible endpoints are supported best-effort through the Chat Completions API.

Who can manage LLM providers

Creating, editing, deleting, verifying, or key-binding LLM providers is not gated to organization admins today: any member of the organization holding an API key can perform these actions. This differs from embedding providers and model credentials, which require the ADMIN role for the same operations.

Configure a provider​

  1. Open Settings → LLM Providers and select New.
  2. Select the provider type: azure-openai, openai, or openai-compatible.
  3. For Azure OpenAI, enter the Azure endpoint and deployment name.
  4. For OpenAI, enter the model name (for example gpt-4o) as the deployment name. DocuDevs uses OpenAI's default API URL, so the API URL field can be left blank.
  5. For OpenAI-compatible, enter the base URL and model name.
  6. Select a shared model credential, or enter an API key to create a dedicated credential, and save the provider.
  7. Bind the provider to one or more levels: DEFAULT, NANO, MINI, or HIGH.

Only explicitly bound levels use the provider. Unbound levels continue to use DocuDevs platform models.

Create a provider​

API_URL="https://api.docudevs.ai"
API_KEY="<your-api-key>"

curl -X POST "$API_URL/llm/providers" \
  -H "Authorization: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Our Azure OpenAI",
    "apiUrl": "https://myacct.openai.azure.com",
    "modelType": "azure-openai",
    "deploymentName": "gpt-4o",
    "credentialId": 42
  }'

Update or delete a provider​

curl -X PATCH "$API_URL/llm/providers/7" \
  -H "Authorization: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"deploymentName": "gpt-4o-mini"}'

curl -X DELETE "$API_URL/llm/providers/7" -H "Authorization: $API_KEY"

Verify credentials​

Run a document-processing job after binding the provider. The real job is the credential check and exercises the same structured-output, vision, streaming, and usage path as production work. Invalid credentials or configuration fail the affected job; DocuDevs does not silently reroute it to a platform model.

You can also probe a provider directly with Verify provider, which calls the configured chat endpoint and checks that the token, gateway authorization, deployment, and model settings work together:

curl -X POST "$API_URL/llm/providers/7/verify" -H "Authorization: $API_KEY"
# -> 202 Accepted

Shared credentials can use either an API key or a Microsoft Entra client ID/client secret. See Model credentials for the Entra app-registration prerequisites, rotation, validation, and rollout constraints. The same shared credential can be selected by chat and embedding providers.

Provider secrets are write-only in the public API and are not returned to the UI. Usage through your own provider is tracked for observability but is not charged against your DocuDevs token balance.

For a shared credential, Validate credential proves that DocuDevs can decrypt an API key or acquire an Entra access token. It does not call the model gateway. Use Verify provider to call the configured chat endpoint and verify that the token, gateway authorization, deployment, and model settings work together. Either check fails closed; DocuDevs does not fall back to a platform model.

Key bindings​

Each logical level (DEFAULT, NANO, MINI, HIGH) is bound to a provider independently. Listing and binding keys use the correctly-mapped list_llm_keys/update_llm_key_binding convenience methods on DocuDevsClient.

curl "$API_URL/llm/keys" -H "Authorization: $API_KEY"

curl -X PUT "$API_URL/llm/keys/HIGH" \
  -H "Authorization: $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"providerId": 7}'

Override reasoning effort for one job​

The Python SDK accepts an optional reasoning_effort keyword on document-processing helpers. It overrides the selected model level's configured effort for the entire job, including pipeline steps and map-reduce chunks:

guid = await client.submit_and_process_document(
    document=document,
    document_mime_type="application/pdf",
    prompt="Extract the contract terms.",
    llm="HIGH",
    reasoning_effort="low",
)

Omit the keyword to retain the built-in model level or BYOM provider default. Accepted values depend on the selected model and provider; blank values are rejected.

In the Java SDK, set the same override with ProcessOptions.builder().reasoningEffort("low").

Embeddings​

Knowledge-base embeddings are configured separately under Settings → Embedding Providers — see Bring your own embeddings. Chat and embedding bindings are independent: changing an LLM provider never changes how your knowledge bases are embedded, and vice versa.