AWS Bedrock

Cold start times vary by model: Claude-3 typically 3-7s, Llama2 8-12s. Once warm, inference is 800ms-2.5s p95 depending on model and token count. No semantic caching layer — each request hits the foundation model. Multi-model routing adds 200-500ms overhead. Cannot achieve consistent sub-2s with current cold start behavior.

Standard REST API with JSON payloads, but model-specific parameters require different request formats. No unified query language — you're still writing model-specific prompts. Documentation covers API mechanics but lacks guidance on model selection logic or prompt optimization across providers. Learning curve moderate due to AWS-specific configuration patterns.

IAM-based access control only — pure RBAC without ABAC capabilities. No row/column-level security for training data. Model access permissions are binary (can/cannot use Claude-3), not contextual (can use for internal docs but not customer data). SOC2 Type II and HIPAA BAA available, but no fine-grained audit of which model processed which query.

Pure AWS lock-in — cannot migrate model configurations, custom fine-tuning, or usage patterns to other clouds. Bedrock-specific APIs mean vendor switching requires full application rewrites. No multi-cloud deployment. Single-region failures take down all models simultaneously. Adaptive capabilities limited to AWS-approved model updates.

Integrates with AWS ecosystem (S3, Lambda, SageMaker) but limited cross-cloud connectivity. No native lineage tracking for model decision flows. CloudTrail captures API calls but not semantic reasoning paths. Model switching logic is opaque — cannot trace why Bedrock selected Claude over Llama for a specific query.

CloudTrail logs API calls but not model reasoning. No query plans or decision trees explaining model selection. Token-level cost attribution exists but no per-query business cost mapping. Cannot trace from final response back through model selection to original data sources. Explainability varies wildly by underlying model provider.

AWS Config and CloudFormation enable policy-as-code for model access controls. VPC endpoints and encryption at rest/transit standard. However, no automated content filtering or toxicity detection across all models — depends on individual provider implementations. Data residency controls through AWS regions but not model-specific data handling policies.

CloudWatch provides basic metrics (latency, error rates, token counts) but lacks LLM-specific observability like hallucination detection or prompt injection monitoring. Third-party APM integration possible but not native. No drift detection for model behavior changes when providers update their models.

99.9% SLA with financial credits. Multi-AZ deployment automatic. RTO typically under 15 minutes for regional failures. Auto-scaling handles traffic spikes without pre-provisioning. Well-established AWS infrastructure reliability patterns with 10+ year track record.

No unified semantic layer — each model provider has different ontology and reasoning patterns. Cannot enforce consistent terminology across Claude vs Llama responses. No business glossary integration or entity resolution across model outputs. Metadata handling varies by underlying provider.

Launched 2023 but built on 17+ years of AWS infrastructure. 100+ enterprise customers in first year. Breaking changes rare due to versioned APIs. AWS guarantees model availability through provider SLAs, but no direct data quality guarantees — that flows through to underlying model providers.