Translate requirements into model, data, retrieval, prompt, and compliance decisions with measurable quality and traceable governance.
Elicit business, quality, latency, cost, privacy, safety, integration, and operational requirements before choosing a model or pattern.
Compare model capability, context, modalities, latency, price, regional availability, customization, and responsible-use constraints.
Assess provenance, permission, representativeness, quality, privacy, transformation, metadata, retention, and safe ingestion.
Choose embeddings, dimensionality, indexing, metadata filters, distance measures, tenancy, update behavior, and access boundaries.
Design chunking, retrieval, reranking, grounding, citations, fallback, freshness, and evaluation for RAG systems.
Use structured instructions, context boundaries, examples, output contracts, templates, versioning, testing, and approval controls.
Define the business task, affected users, acceptable behavior, failure cost, privacy boundary, quality rubric, latency, throughput, budget, integration, availability, human oversight, and evidence needs before comparing models. “Use generative AI” is not a requirement.
Model selection should compare task performance on representative data, modalities, context limits, structured output, tool behavior, regional availability, latency, price, customization, safety, and operational support. A larger model is not automatically the best production choice.
Confirm provenance and permission before ingestion. Validate format, language, duplication, sensitivity, representativeness, malicious content, retention, ownership, and update behavior. Chunking, embeddings, vector index, metadata, filtering, reranking, generation, citations, and deletion are distinct control and failure layers.
RAG is appropriate when the system needs fresh or private knowledge with traceable sources. Prompting changes instructions and context. Fine-tuning can change stable behavior but introduces training-data, evaluation, versioning, safety, cost, and lifecycle responsibilities. Use the least invasive pattern that satisfies measured requirements.
Version system instructions, templates, examples, variables, retrieval boundaries, output schemas, safety rules, and evaluation results. Treat retrieved text and user input as untrusted data, not higher-priority instructions. Enforce authorization outside the model.
Defend a model, chunking, retrieval, and prompt design for a regulated knowledge assistant. State what would make you choose a different model or replace RAG with another pattern.
