LLM workflows

Generation and config

Generation turns source notes into selected standards artifacts. Keep each source folder aligned with the selected --kind, copy config before editing, and validate generated YAML before downstream use.

Copy config and prompts

The copied config becomes the file you pass to generation with --config. Copying it does not generate anything by itself. The useful flow is: create project folders, copy the bundled config, edit the project copy, check it, run generation with that config, then validate the YAML that was written to the configured output folder.

Config workflowcopy, inspect, check

mkdir -p config prompts source_docs/signals generated/fragments generated/catalogs

open-data-products config generation --copy-to config/generation.config.yaml
open-data-products config generation --copy-prompts-to prompts/

# Edit config/generation.config.yaml before running generation.
open-data-products config generation --config config/generation.config.yaml --print
open-data-products config generation --config config/generation.config.yaml --check
open-data-products config generation --config config/generation.config.yaml --check --json

In the copied file, set the top-level provider to the provider profile you want to use by default. Set input to the folder or file that contains source notes, and set output to the folder where generated YAML should be written. The command can still override these values for one run with --provider, --model, --input, and --output.

Config field	Example value	What happens later
`provider`	`ollama-qwen3`	`generate --config config/generation.config.yaml` uses the matching entry under `providers`.
`input`	`source_docs/signals/`	Generation reads every supported source file in that folder unless `--input` overrides it.
`output`	`generated/fragments/`	Generated ODPC fragment YAML is written there, then validated or assembled into a catalog.
`providers.ollama-qwen3.model`	`qwen3:8b`	The provider adapter sends the prompt to that model when `ollama-qwen3` is selected.
`providers.openai.apiKeyEnv`	`OPENAI_API_KEY`	The SDK reads the secret from the environment variable at runtime; the YAML stores only the variable name.

Do not store secrets in YAML. Provider entries store the environment variable name in apiKeyEnv. Export the actual key in the environment before running generation.

Use the copied config

The source file below is plain text. The --kind signal flag tells the SDK which prompt and output shape to use. The copied config supplies the default provider, model, input folder, and output folder; the command below overrides provider, input, and output for one controlled test run.

One complete local runsource to validated fragment

cat > source_docs/signals/turnaround-delay-signal.txt <<'TXT'
Airport operations teams need an early signal when aircraft turnaround is at
risk of exceeding planned ground time. The signal should consider late inbound
arrival, missing cleaning crew assignment, delayed baggage unloading, and gate
conflict status.
TXT

ollama pull qwen3:8b

open-data-products generate \
  --config config/generation.config.yaml \
  --provider ollama-qwen3 \
  --input source_docs/signals/turnaround-delay-signal.txt \
  --kind signal \
  --output generated/fragments/ \
  --json

ls generated/fragments/
open-data-products validate generated/fragments/<generated-signal>.yaml
open-data-products explain generated/fragments/<generated-signal>.yaml --json

The result is not a report about the source file. It is a standards YAML artifact, usually named from the generated object id, such as signal_turnaround-delay-risk.yaml. Use that YAML as input to the next workflow: validate it for quality, build it into an ODPC catalog, include it in a portfolio workspace, or use it as graph context.

Use the generated fragment nextcatalog sidecars

open-data-products odpc-build generated/fragments/ \
  --output generated/catalogs/catalog.yaml \
  --html generated/catalogs/catalog.html \
  --toon generated/catalogs/catalog.toon \
  --gcf generated/catalogs/catalog.gcf \
  --json

open-data-products odpc-summary generated/catalogs/catalog.yaml --json

Online provider variant

For an online provider, the generated file is used the same way. The only difference is the selected provider profile and the exported environment variable. If the copied config contains apiKeyEnv: OPENAI_API_KEY, export OPENAI_API_KEY before running generation. For Anthropic, export the variable named by the selected Anthropic profile.

OpenAI-compatible runsame output workflow

export OPENAI_API_KEY="..."

open-data-products generate \
  --config config/generation.config.yaml \
  --provider openai \
  --model gpt-4.1-mini \
  --input source_docs/signals/turnaround-delay-signal.txt \
  --kind signal \
  --output generated/fragments/ \
  --json

open-data-products validate generated/fragments/<generated-signal>.yaml

Prepare source files

The generator reads Markdown and text files directly. It can also include YAML catalog or graph context when the YAML file has a sibling .toon or .gcf sidecar. Keep folder inputs type-specific so one prompt does not receive mixed product, signal, use-case, and objective material.

Source examplessmall text inputs

source_docs/
  products/
    customer-retention-product.md
  signals/
    churn-risk-signal.txt
  use-cases/
    renewal-intervention-use-case.md
  objectives/
    reduce-enterprise-churn.txt

Generate one artifact

Use one file first when testing a provider or prompt. The output file name is derived from the generated artifact id for fragment kinds, so validate the generated folder rather than assuming the output filename will match the source filename exactly.

Single sourceselected artifact kind

open-data-products generate \
  --input source_docs/signals/turnaround-delay-signal.txt \
  --kind signal \
  --output generated/fragments/ \
  --model qwen2.5 \
  --json

open-data-products validate generated/fragments/<generated-signal>.yaml

Generate from a folder

For folder generation, put only matching source files in the folder. If --kind signal points at a mixed folder containing products, objectives, and signal notes, the selected signal prompt receives the wrong evidence.

Folder sourcematching source files only

open-data-products generate \
  --config generation.config.yaml \
  --input source_docs/products/ \
  --kind product-reference \
  --provider ollama-gemma3n \
  --output generated/fragments/ \
  --json

Supported generation kinds are product-reference, odps-product, use-case, objective, signal, and graph.

Kind	Typical input	Typical output	Use next
`product-reference`	Product brief, system description, dataset notes.	ODPC `productReference` fragment.	`odpc-build` or portfolio build.
`odps-product`	Full product narrative or product spec notes.	ODPS product YAML.	`validate`, contract checks, catalog inclusion.
`use-case`	User journey, decision workflow, stakeholder need.	ODPC `useCase` fragment.	Portfolio build and graph context.
`objective`	Business target, OKR, strategic goal.	ODPC `businessObjective` fragment.	Portfolio build and graph context.
`signal`	Risk, demand, market, usage, or quality evidence.	ODPC `signal` fragment.	Portfolio build and graph context.
`graph`	Fragments, catalog context, or graph notes.	ODPG graph YAML.	`odpg-summary`, traversal, graph explorer.

ODPS product profiles

Product draftminimal or complete

open-data-products generate \
  --input source_docs/products/revenue-product.md \
  --kind odps-product \
  --profile minimal \
  --output generated/products/ \
  --json

open-data-products generate \
  --input source_docs/products/revenue-product.md \
  --kind odps-product \
  --profile complete-draft \
  --include-components SLA,dataQuality,pricingPlans,dataAccess,productStrategy \
  --output generated/products/ \
  --json

Provider configuration

The selected top-level provider points to one entry in providers. CLI flags such as --provider, --model, --input, and --output override the file for one run. Use overrides for testing; put stable team defaults in the config.

YAMLsafe provider profile

provider: enterprise-gateway
input: source_docs/products/
output: generated/fragments/

providers:
  enterprise-gateway:
    type: openai-chat
    model: data-product-generation
    baseUrl: https://gateway.example.com/v1
    apiKeyEnv: ENTERPRISE_LLM_API_KEY

portfolio:
  sourceBudget:
    maxSourceChars: 2000
    maxPromptChars: 32000
  privacy:
    obfuscatePersonalData: true

Type	Use for	Keys
`ollama`	Local Ollama models.	`model`, `baseUrl`
`openai-chat`	OpenAI-compatible chat completions, including LM Studio, vLLM, NVIDIA NIM, llama.cpp server, LocalAI, hosted chat providers, and enterprise gateways.	`model`, `baseUrl`, `apiKeyEnv`
`openai`	Responses API compatible profiles such as OpenAI, OpenRouter, Groq, and Sakana Fugu.	`model`, `baseUrl`, `apiKeyEnv`
`anthropic`	Claude through Anthropic Messages API.	`model`, `baseUrl`, `apiKeyEnv`, `version`, `maxTokens`
`llama-cpp`	Direct local GGUF inference through `llama-cpp-python`.	`modelPath`, `contextWindow`, `gpuLayers`

AI Gateway

Use the normal generation config for an AI Gateway. The SDK does not need a separate gateway config when the gateway exposes an OpenAI-compatible chat endpoint. Point baseUrl to the gateway, set type to openai-chat, and set model to the gateway route or workload alias. The SDK sends that alias in the request. The gateway decides which real provider and model handle the request, and can apply its own policy, fallback, logging, budget, and key management.

Gateway configroute alias instead of direct model ownership

provider: ai-gateway
input: source_docs/products/
output: generated/products/

providers:
  ai-gateway:
    type: openai-chat
    model: data-product-generation
    baseUrl: https://ai-gateway.example.com/v1
    apiKeyEnv: AI_GATEWAY_API_KEY

In this example, data-product-generation is not a hard-coded vendor model in the SDK. It is the route name your gateway understands. The gateway may map it to a hosted model, a local model, a fallback chain, or a cost-controlled policy. Keep that mapping in the gateway, not in the SDK docs project.

Gateway runSDK calls the route

export AI_GATEWAY_API_KEY="..."

open-data-products config generation --config config/generation.config.yaml --check

open-data-products generate \
  --config config/generation.config.yaml \
  --provider ai-gateway \
  --input source_docs/products/revenue-product.md \
  --kind odps-product \
  --profile minimal \
  --output generated/products/ \
  --json

open-data-products validate generated/products/<generated-product>.yaml

Concern	Owned by SDK config	Owned by gateway
Where requests go	`baseUrl` points to the gateway endpoint.	Gateway receives the OpenAI-compatible request.
Which workload is requested	`model` carries the route alias, such as `data-product-generation`.	Gateway maps that alias to actual model policy.
Secrets	`apiKeyEnv` names the environment variable.	Gateway can hold downstream provider keys.
Provider selection	Not hard-coded when using a gateway route.	Gateway chooses provider, model, fallback, retries, and budget controls.

Temporary overrides

CLI flagsone run only

export OPENAI_API_KEY="..."

open-data-products generate \
  --config generation.config.yaml \
  --provider openai \
  --model gpt-4.1-mini \
  --input source_docs/signals/ \
  --kind signal \
  --output generated/openai-test/ \
  --json

Local runtime examples

Ollamalocal model path

ollama pull qwen3:8b

open-data-products generate \
  --config generation.config.yaml \
  --provider ollama-qwen3 \
  --input source_docs/signals/ \
  --kind signal \
  --output generated/fragments/ \
  --json

LM Studio or gatewayOpenAI-compatible chat

open-data-products generate \
  --config generation.config.yaml \
  --provider lmstudio \
  --model local-model \
  --input source_docs/products/ \
  --kind product-reference \
  --output generated/fragments/ \
  --json

Validation loop

Run generation with one small source file.
Validate generated YAML with open-data-products validate generated/fragments/artifact.yaml.
Explain the artifact with open-data-products explain generated/fragments/artifact.yaml --json.
Only then build ODPC catalogs, ODPG graphs, or portfolio outputs.