Why Self-Host Instead of Using APIs

API-based LLMs from OpenAI, Anthropic, and Google are easier to use—no infrastructure management, automatic updates, and predictable scaling. Self-hosting is harder but provides advantages that justify the complexity in specific scenarios.

Cost reduction at scale is the primary driver. Running a 7B parameter model on a single GPU costs $300-800/month for cloud hosting or $2,000-5,000 upfront for on-premise hardware. That infrastructure can serve hundreds of thousands to millions of queries depending on your latency requirements. Compare to API costs where you pay per token—the break-even point is typically 100,000-500,000 queries monthly. Above that volume, self-hosting is dramatically cheaper.

Data privacy and compliance requirements force self-hosting for many enterprises. If you're processing healthcare records (HIPAA), financial data (SOC 2), or EU citizen data (GDPR), sending it to third-party APIs creates compliance risks. Self-hosting keeps data within your infrastructure boundary, simplifying compliance and reducing legal liability. Some organizations are prohibited from using external LLMs entirely—self-hosting is the only option.

The Real Costs of Self-Hosting

Self-hosting isn't just infrastructure costs—it's engineering time. You need to select models, set up serving infrastructure, implement monitoring, handle model updates, and debug issues when performance degrades. For a small team, this can consume 20-40% of one engineer's time. Factor this into your cost analysis: if an engineer costs $150k/year, you're spending $30-60k annually on maintenance.

The infrastructure itself has surprising complexity. A single GPU server seems simple until you need high availability (requiring at least two servers and a load balancer), automatic scaling (requiring orchestration), and monitoring (requiring observability infrastructure). What starts as "just run a model on a GPU" becomes a full ML infrastructure project. Managed services like AWS SageMaker or HuggingFace Inference Endpoints reduce this complexity but cost 2-3x more than bare GPU instances.

Performance Expectations

Open-source models lag behind frontier models on most benchmarks. The best open-source models (LLaMA 3 70B, Mixtral 8x7B) approximate GPT-4's quality on many tasks but still fall short on complex reasoning, coding, and instruction following. Models in the 7-13B range approximate GPT-3.5 quality. Smaller models (1-3B) are useful for specific narrow tasks but can't match larger models on general capabilities.

The practical implication: you trade some quality for cost savings and data control. For many applications, this trade is acceptable—customer support doesn't need perfect responses, code completion works fine with good-enough suggestions, and content generation can have human review. For applications requiring frontier model quality, self-hosting isn't viable yet; stick with GPT-4 or Claude.

LLaMA 3: Meta's Open-Source Flagship

Meta's LLaMA 3 is the current gold standard for open-source LLMs. Released in April 2024, it significantly improved on LLaMA 2, matching or exceeding GPT-3.5 quality across most benchmarks. LLaMA 3 comes in 8B and 70B parameter sizes, both with instruct-tuned variants optimized for following instructions.

Model Variants and Capabilities

LLaMA 3 8B requires 16GB VRAM for inference (8-bit quantization) or 32GB for full precision. It handles general text generation, simple coding tasks, and question answering with quality approximating GPT-3.5. The 70B model requires 80GB VRAM (A100) for 8-bit inference or 140GB for full precision, delivering quality that approaches GPT-4 on many tasks. Both models support 8k token context windows, sufficient for most applications.

The instruct-tuned variants (LLaMA-3-8B-Instruct, LLaMA-3-70B-Instruct) are trained to follow instructions in chat format. These are what you want for most applications—the base models require specific prompting techniques and are harder to work with. The instruct models understand system prompts, maintain conversation context, and follow complex instructions reliably.

# Running LLaMA 3 with vLLM
from vllm import LLM, SamplingParams

# Load the 8B instruct model with 8-bit quantization
llm = LLM(
    model="meta-llama/Meta-Llama-3-8B-Instruct",
    quantization="awq",  # 8-bit quantization
    dtype="half",
    gpu_memory_utilization=0.9
)

sampling_params = SamplingParams(
    temperature=0.7,
    top_p=0.9,
    max_tokens=2048
)

prompts = [
    "Write a Python function to calculate fibonacci numbers"
]

outputs = llm.generate(prompts, sampling_params)
for output in outputs:
    print(output.outputs[0].text)

When LLaMA 3 Excels

LLaMA 3 8B is ideal for applications needing GPT-3.5-level quality at much lower cost. Customer support chatbots, content generation, summarization, and simple coding assistance work well. In benchmarking on customer support responses, LLaMA 3 8B achieved 88% of GPT-3.5's helpfulness scores while running on single-GPU infrastructure costing $0.002 per query vs $0.02 for GPT-3.5 API calls—a 10x cost reduction.

LLaMA 3 70B competes with GPT-4 on tasks like complex reasoning, multi-step problem solving, and advanced code generation. It's particularly strong on mathematical reasoning and logical deduction. For applications where GPT-4 works but cost or data privacy prevents API usage, LLaMA 3 70B is the best open-source alternative. The infrastructure cost is significant—you need A100 GPUs—but for high-volume applications, the per-query cost is still 10-50x lower than GPT-4 APIs.

Limitations

LLaMA 3 underperforms frontier models on nuanced instruction following, creative writing, and handling ambiguous queries. It's more likely to misinterpret unclear instructions or give overly literal responses. The 8k context limit is restrictive for document analysis—GPT-4 Turbo's 128k context lets you process entire documents, while LLaMA 3 requires chunking.

License restrictions complicate commercial use. LLaMA 3's license allows commercial use but with restrictions on training competing models and requirements to comply with Meta's acceptable use policy. For most SaaS applications, this is fine, but read the license carefully to ensure compliance with your use case.

Mistral and Mixtral: Efficient European Models

Mistral AI, a French startup, released models that punch above their weight class. Mistral 7B delivers quality approaching 13B models from competitors, and Mixtral 8x7B uses a mixture-of-experts architecture to achieve near-70B quality with 13B active parameters per query.

Mistral 7B

Mistral 7B v0.3 is one of the most efficient small models available. It matches or exceeds LLaMA 2 13B on most benchmarks while requiring only 14GB VRAM (8-bit) for inference. The instruct version (Mistral-7B-Instruct-v0.3) handles chat interactions well and follows instructions more reliably than similarly-sized models.

The key innovation is training efficiency. Mistral uses techniques like sliding window attention to process longer contexts effectively. The model supports 32k token context with 8k sliding window, meaning it can attend to recent context with full attention and earlier context with sparse attention. This enables document Q&A and summarization tasks that would overflow smaller context windows.

Mixtral 8x7B

Mixtral uses 8 expert models (each 7B parameters) with a router that activates 2 experts per token. This sparse activation means only 13B parameters are active for any query, dramatically reducing inference cost compared to dense 50-70B models. The effective capacity is much larger—47B total parameters—giving it quality that competes with LLaMA 3 70B on many tasks.

Mixtral requires 90GB VRAM (8-bit quantization) for inference, fitting on a single A100 80GB GPU with some spillover to CPU memory or requiring distributed inference across 2 GPUs. The inference speed is 2-3x faster than dense 70B models because fewer parameters activate per token. For applications where GPT-4-level quality is needed but latency matters, Mixtral provides better throughput than dense alternatives.

# Mixtral inference with model parallelism
from vllm import LLM, SamplingParams

llm = LLM(
    model="mistralai/Mixtral-8x7B-Instruct-v0.1",
    tensor_parallel_size=2,  # Distribute across 2 GPUs
    quantization="awq",
    dtype="half"
)

# Mixtral excels at complex reasoning
prompts = [
    "Explain the pros and cons of microservices vs monolithic architecture for a 10-person startup building a SaaS product."
]

outputs = llm.generate(prompts, SamplingParams(temperature=0.7, max_tokens=1024))
print(outputs[0].outputs[0].text)

When to Choose Mistral Models

Mistral 7B is the best small model for most use cases. If you're constrained to single consumer GPU (RTX 4090, RTX 3090) or need very fast inference, Mistral 7B delivers quality approaching much larger models. It's ideal for customer support, simple coding assistance, content generation, and any application where GPT-3.5 quality suffices but you need low latency or low cost.

Mixtral 8x7B competes with LLaMA 3 70B at lower infrastructure cost (can run on 2x mid-tier GPUs vs requiring A100s) and higher throughput. For applications needing strong reasoning, complex instructions, or code generation, Mixtral provides the best quality-to-cost ratio in open-source models. Testing on code explanation tasks showed Mixtral matching GPT-4 quality on 70% of queries while running on infrastructure 5x cheaper than GPT-4 API costs.

License and Availability

Mistral models use Apache 2.0 license, allowing unrestricted commercial use. This is more permissive than LLaMA's license, making Mistral models safer for businesses concerned about licensing restrictions. The models are available on HuggingFace and integrate seamlessly with standard inference libraries like vLLM and TGI.

Specialized Models: Code, Math, and Domain-Specific

General-purpose models work across tasks but specialized models trained on domain-specific data outperform on their specialty. If your application focuses heavily on code, math, or a specific domain, specialized models provide better results at similar or lower infrastructure cost.

DeepSeek Coder: Code Generation Excellence

DeepSeek Coder 33B is trained specifically on code and technical documentation. It outperforms general models on code generation, code explanation, and debugging tasks. The model understands code context across multiple files and can follow repository structure, making it ideal for coding assistants and automated code review.

DeepSeek Coder requires 40GB VRAM (8-bit) for the 33B variant, fitting on A100 40GB or distributed across 2x RTX 4090. In benchmarking on HumanEval (code generation benchmark), DeepSeek Coder 33B scored 75% compared to GPT-4's 85% and GPT-3.5's 48%. For a self-hosted model, this is exceptional—you get near-GPT-4 code quality at a fraction of the cost.

# DeepSeek Coder for code completion
llm = LLM(
    model="deepseek-ai/deepseek-coder-33b-instruct",
    quantization="awq",
    dtype="half"
)

prompt = """Complete this Python function:

def calculate_moving_average(data, window_size):
    '''
    Calculate moving average of a list of numbers.
    data: list of numbers
    window_size: integer window size
    returns: list of moving averages
    '''"""

output = llm.generate(prompt, SamplingParams(temperature=0.2, max_tokens=512))
print(output[0].outputs[0].text)

CodeLlama: Meta's Code Specialist

CodeLlama is Meta's code-specialized variant of LLaMA 2, trained on 500B tokens of code. The 34B variant provides strong code generation at lower resource requirements than DeepSeek Coder. CodeLlama also offers Python-specific variants (CodeLlama-Python) that excel at Python code generation and understanding.

CodeLlama's advantage is the instruct-tuned variants that combine code generation with instruction following. You can ask it to generate code, explain code, debug errors, or refactor implementations in natural language. For developers building AI coding assistants, CodeLlama provides a strong foundation that can be fine-tuned on your codebase for even better context-aware suggestions.

WizardMath and MAmmoTH: Mathematical Reasoning

If your application involves mathematical problem solving, specialized math models significantly outperform general models. WizardMath 70B and MAmmoTH 70B are fine-tuned on mathematical reasoning tasks and achieve scores approaching or exceeding GPT-4 on math benchmarks like GSM8K and MATH.

These models understand mathematical notation, multi-step problem solving, and can show their work step-by-step. For educational applications, homework helpers, or technical applications involving calculations and proofs, math-specialized models provide accuracy that general models can't match. The infrastructure requirement is similar to LLaMA 3 70B (80GB VRAM), but the quality on math tasks is substantially higher.

Quantization: Trading Quality for Speed and Cost

Quantization reduces model precision from 16-bit floats to 8-bit or 4-bit integers, shrinking model size and memory requirements by 2-4x. This enables running larger models on cheaper GPUs or running models faster on the same hardware. The trade-off is quality loss—quantized models are slightly less accurate than full-precision versions.

Quantization Methods

8-bit quantization (LLM.int8()) reduces VRAM requirements by roughly 50% with minimal quality loss—typically under 1% accuracy degradation on benchmarks. This is the safe default for production systems. A 7B model drops from 14GB to 7GB, a 70B model from 140GB to 70GB. Most models run fine with 8-bit quantization.

4-bit quantization (GPTQ, AWQ) reduces VRAM by 75%, enabling a 7B model in 3.5GB and a 70B model in 35GB. Quality loss is more noticeable—2-5% accuracy degradation—but for many applications, the speed and cost benefits outweigh the quality trade-off. 4-bit quantization is what enables running 70B models on consumer hardware (2x RTX 4090 48GB).

# Loading models with different quantization
from vllm import LLM

# 8-bit quantization (recommended)
llm_8bit = LLM(
    model="meta-llama/Meta-Llama-3-70B-Instruct",
    quantization="awq",  # 8-bit AWQ quantization
    dtype="half"
)

# 4-bit quantization (more aggressive)
llm_4bit = LLM(
    model="meta-llama/Meta-Llama-3-70B-Instruct",
    quantization="gptq",  # 4-bit GPTQ quantization
    dtype="half"
)

# VRAM requirements:
# 8-bit: ~70GB (fits A100)
# 4-bit: ~35GB (fits 2x RTX 4090)

When to Use Aggressive Quantization

4-bit quantization makes sense when infrastructure cost is the primary constraint or when you need to run large models on consumer hardware for development. The quality loss is acceptable for many practical applications—chatbots, content generation, simple coding assistance—where perfect accuracy matters less than response speed and cost efficiency.

Avoid aggressive quantization for applications requiring high accuracy on edge cases, complex reasoning, or mathematical precision. In testing on coding benchmarks, 4-bit quantized models showed 5-8% lower pass rates compared to 8-bit versions. For production systems where quality is critical, stick with 8-bit quantization or full precision if your infrastructure can handle it.

Infrastructure Requirements and Costs

Understanding infrastructure requirements helps you budget and choose the right model for your constraints. Here's what you need for different deployment scales.

Single GPU Development Setup

For development and low-traffic production (under 100 queries/day), a single consumer GPU works. An RTX 4090 (24GB VRAM) runs 7B models comfortably with 8-bit quantization, handling Mistral 7B or LLaMA 3 8B with room to spare. Cost: $1,600-2,000 hardware or $1.50/hour cloud (AWS g5.xlarge).

This setup supports 1-5 concurrent users with 1-3 second response times. Throughput is limited—maybe 20-50 queries per minute depending on output length. For MVPs, internal tools, or research projects, this is sufficient. Production applications with real traffic need more capacity.

Production Single-GPU Setup

For moderate production traffic (1,000-10,000 queries/day), use cloud A10G or A100 GPUs. An A10G (24GB) costs $1.00-1.50/hour and runs 7B models with good throughput (50-100 qpm). An A100 40GB costs $3-4/hour and runs 7-13B models or quantized 33B models, handling 100-300 qpm.

At this scale, implement batching and caching. vLLM and TGI batch requests automatically, significantly improving throughput. Cache common queries or use semantic caching to avoid re-generating similar responses. These optimizations can reduce compute needs by 30-50% for typical workloads.

Multi-GPU for Large Models

Running 70B models requires multi-GPU setups. Two A100 40GB GPUs (tensor parallelism) run 70B models with 4-bit quantization at good speeds. Cost: $6-8/hour cloud or $20k+ for on-premise hardware. Throughput depends on tensor parallel efficiency but expect 20-50 qpm with sub-3-second latencies.

For high-volume production (100,000+ queries/day), use multiple inference replicas with load balancing. Run 3-5 GPU servers behind a load balancer, scaling horizontally as traffic increases. This provides redundancy and handles traffic spikes. At this scale, consider Kubernetes with GPU node pools for autoscaling based on queue depth.

# vLLM serving with horizontal scaling
# Server 1, 2, 3 each running:
python -m vllm.entrypoints.openai.api_server \
  --model meta-llama/Meta-Llama-3-70B-Instruct \
  --tensor-parallel-size 2 \
  --quantization awq \
  --port 8000

# Load balancer distributes across servers
# Kubernetes HPA scales replicas based on metrics

Cost Comparison

The economics favor self-hosting at scale. For 1 million queries/month with 500 token average outputs:

GPT-4 API: $15,000/month (at $0.03/1k output tokens). GPT-3.5 API: $1,500/month (at $0.002/1k tokens). Self-hosted LLaMA 3 8B: $720/month (24/7 A10G at $1/hour). Self-hosted LLaMA 3 70B: $4,320/month (24/7 2x A100 at $6/hour).

The break-even is around 100k queries/month for 8B models and 300k queries/month for 70B models compared to GPT-3.5. Compared to GPT-4, self-hosting breaks even much earlier. These calculations assume you're not over-provisioning—if your infrastructure sits idle 50% of the time, double the costs or implement autoscaling.

Deployment and Serving Best Practices

Deploying models is more than starting a server. Production deployments require serving infrastructure, monitoring, and operational procedures to maintain reliability.

Serving Infrastructure

Use vLLM or Text Generation Inference (TGI) for serving. These frameworks implement optimizations that dramatically improve throughput: continuous batching (dynamically batch requests as they arrive), PagedAttention (efficient memory management), quantization support, and tensor parallelism for multi-GPU. Hand-rolled serving with transformers library is 3-10x slower.

vLLM provides an OpenAI-compatible API, making migration from OpenAI APIs trivial—change the endpoint URL and API key, everything else stays the same. This compatibility is valuable for gradually migrating from APIs to self-hosted models without rewriting application code.

# vLLM OpenAI-compatible server
from openai import OpenAI

# Point at your self-hosted vLLM server
client = OpenAI(
    api_key="dummy-key",
    base_url="http://your-server:8000/v1"
)

# Same interface as OpenAI
completion = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3-8B-Instruct",
    messages=[
        {"role": "user", "content": "Explain quantum computing"}
    ]
)

print(completion.choices[0].message.content)

Monitoring and Observability

Track key metrics: query per second (QPS), latency (p50, p95, p99), GPU utilization, and queue depth. High GPU utilization (>80%) means you're efficiently using resources. Low utilization suggests over-provisioning or batching issues. Rising queue depth indicates insufficient capacity—time to scale up.

Monitor model quality with sampling. Log 1-5% of requests and responses, periodically review for quality issues. Models can degrade due to prompt injections, unexpected input patterns, or infrastructure issues causing corrupted outputs. Regular manual review catches these problems before they affect many users.

Implement fallback to API-based models for reliability. If your self-hosted infrastructure fails, route traffic to OpenAI or Anthropic APIs temporarily. This prevents outages from destroying user experience. The fallback costs money but rarely activates if your primary infrastructure is reliable—acceptable insurance against downtime.

Model Updates and Versioning

New model versions release regularly (LLaMA 3.1, Mistral v0.4, etc.). Evaluate new versions on your test set before deploying. Sometimes new versions regress on specific tasks despite overall improvements. Run A/B tests comparing old and new versions in production to ensure improvements transfer to your use case.

Maintain multiple model versions simultaneously. Deploy new versions alongside old ones, gradually shifting traffic as you gain confidence. This blue-green deployment pattern enables quick rollback if new versions underperform. Use feature flags or traffic routing rules to control which users see which model version.

Choosing the Right Model for Your Use Case

With dozens of open-source models available, selection requires matching model capabilities to your requirements. Here's a decision framework.

Start With Your Constraints

Hardware budget determines model size. If you're limited to single consumer GPU (under $2k), you're choosing between 7-13B models. With cloud budget of $1k/month, you can run 7B models continuously or 70B models for limited hours. With enterprise budget ($10k+/month), any model is viable—choose based on quality needs.

Quality requirements narrow the field. If GPT-3.5 quality suffices, 7-13B models work (Mistral 7B, LLaMA 3 8B). If you need GPT-4-level quality, you need 70B+ models (LLaMA 3 70B, Mixtral 8x7B). If even GPT-4 isn't good enough, self-hosting isn't viable yet—stick with APIs or wait for better open-source models.

Match Model to Task Type

For code generation, use CodeLlama or DeepSeek Coder rather than general models. The specialized training provides 20-30% better accuracy on coding tasks. For customer support, general instruction-tuned models (Mistral 7B Instruct, LLaMA 3 8B Instruct) work well—fine-tune on your support history for best results.

For document Q&A with RAG, context window size matters. Models with 32k+ context (Mistral, newer LLaMA variants) handle larger chunks and more retrieved documents. For creative writing or content generation, models trained on diverse internet text (LLaMA, Mistral) outperform code-focused models.

Frequently Asked Questions

Can open-source models match GPT-4 quality?

On specific tasks, yes. LLaMA 3 70B and Mixtral 8x7B approach GPT-4 performance on coding, reasoning, and many NLP tasks. But GPT-4 remains superior on complex multi-step reasoning, creative writing, and nuanced instruction following. The gap is closing—models released in 2024 are significantly better than 2023 models—but frontier models still lead. For many practical applications, "90% of GPT-4 quality at 5% of the cost" is a worthwhile trade-off.

What's the smallest usable model size?

For general chatbot applications, 7B is the practical minimum. Smaller models (1-3B) work for very specific narrow tasks where you can fine-tune heavily, but struggle with general instruction following. The sweet spot for cost vs capability is 7-13B models like Mistral 7B or LLaMA 3 8B. These run on consumer hardware while providing quality acceptable for most production use cases.

How do I fine-tune self-hosted models?

Fine-tuning open-source models is easier than closed models. Use HuggingFace PEFT for LoRA fine-tuning, which works on most models. The process: prepare training data in instruction format, configure LoRA parameters (rank, alpha, target modules), train with HuggingFace Trainer, save adapter weights, and load them alongside base model for inference. For 7B models, fine-tuning takes 1-4 hours on single GPU depending on dataset size.

What about model licenses and commercial use?

Licenses vary. Mistral models use Apache 2.0 (fully permissive). LLaMA 3 allows commercial use but with restrictions (check Meta's license). Some models have non-commercial licenses (research only). Always read the license before deploying commercially. For businesses, Mistral's Apache 2.0 license or models explicitly marked "commercial use allowed" are safest. Consult legal counsel for high-stakes deployments.

How often should I update to newer model versions?

Evaluate new versions quarterly but don't update blindly. New releases often improve benchmarks but may regress on your specific use case. Test new versions on your evaluation set. If they show 5%+ improvement on metrics you care about, run A/B tests in production. If production metrics improve, gradually roll out the new version. Many teams stick with versions that work well for 6-12 months unless compelling improvements emerge.

Can I run multiple models simultaneously for different tasks?

Yes, but memory is the constraint. Running two 7B models requires 2x the VRAM (28-32GB with quantization), typically needing an A100 or multiple consumer GPUs. A better approach for multi-task serving: use a routing layer that loads models on-demand. Keep the most-used model in memory and swap others as needed, or use separate servers for different models with a gateway routing requests to the appropriate server.

What happens if my self-hosted infrastructure goes down?

Implement automatic fallback to API-based models. When health checks fail on your self-hosted endpoint, route traffic to OpenAI or Anthropic APIs. This prevents complete outages. The fallback will be more expensive but rarely activates if your infrastructure is reliable. For critical applications, run redundant instances across multiple availability zones—if one fails, others continue serving traffic.

How do I handle model updates without downtime?

Use blue-green deployment. Run the new model version alongside the old one on separate infrastructure. Gradually shift traffic (5%, 25%, 50%, 100%) to the new version while monitoring metrics. If issues appear, roll back by shifting traffic back to the old version. Once the new version proves stable, decommission the old one. This requires 2x infrastructure temporarily but eliminates deployment risk.

Can I use consumer GPUs for production serving?

For low-traffic applications (under 10,000 queries/day), consumer GPUs (RTX 4090, 4080) work fine. They run 7B models comfortably. The downsides: no ECC memory (higher error risk), shorter warranty, and lower reliability than datacenter GPUs. For mission-critical production with high traffic, use datacenter GPUs (A10G, A100, H100) that are designed for 24/7 operation with better reliability guarantees.

How much does self-hosting actually save compared to APIs?

It depends on volume. Below 50,000 queries/month, APIs are cheaper when you factor in engineering time. Between 50k-500k queries/month, savings are modest (30-50%). Above 500k queries/month, savings are substantial (70-90%). For 1 million queries/month at 500 tokens output, GPT-3.5 costs $1,000/month while self-hosted 8B model costs $200-300/month including infrastructure. The break-even point is typically 100-200k queries/month.

Conclusion

Open-source LLMs have reached the point where self-hosting is viable for production applications. Models like LLaMA 3 8B approximate GPT-3.5 quality while running on affordable hardware, and LLaMA 3 70B approaches GPT-4 on many tasks at a fraction of API costs. Specialized models like DeepSeek Coder deliver near-frontier performance on coding tasks. The infrastructure and operational complexity is real, but for applications with sufficient query volume or strict data privacy requirements, the economics clearly favor self-hosting.

The decision framework is straightforward: start with your constraints (infrastructure budget, quality requirements, data privacy needs), match them to available models, and prototype with the most promising candidates. Test extensively on representative workloads before committing—benchmark performance, quality, and costs against API alternatives. For most teams, the entry point is small models (7-13B) for specific tasks, with options to scale to larger models as volume and budget grow.

The open-source model landscape evolves rapidly. Models that were state-of-the-art six months ago are now mid-tier. Track releases from Meta, Mistral, DeepSeek, and the broader HuggingFace community. As models improve and infrastructure costs decrease, the set of applications where self-hosting makes sense will continue expanding.