1.3 KiB
1.3 KiB
AI and LLM Workloads
- Treat prompts, schemas, model routing, retrieval, pre-processing, post-processing, thresholds, and evaluations as source-controlled behavior.
- Explain the behavioral impact before changing a provider, model, embedding, prompt, threshold, chunking, or ranking strategy.
- Do not call paid model, embedding, transcription, vision, or other metered APIs without separate explicit approval.
- Do not send proprietary, personal, customer, survey, receipt, transcript, or other sensitive data to an external model or service without exact authorization for that data and destination.
- Prefer mocks and sanitized deterministic fixtures for development and tests.
- Validate structured model output before business use and handle refusals, malformed output, timeouts, rate limits, and provider failures explicitly.
- Keep API keys and provider credentials in the repository's approved secret mechanism, never in source, prompts, fixtures, logs, or committed configuration.
- Redact sensitive prompt, response, token, trace, and provider-error content from logs and reports.
- Separate deterministic business rules from probabilistic model behavior so critical rules remain testable.
- Compare representative evaluation results before and after behavioral changes, and report material quality, cost, latency, privacy, and reproducibility tradeoffs.