Provider Configuration

Providers are external AI services that Admin Bud-E connects to for LLM, VLM, TTS, and ASR capabilities. Each provider requires:

• A unique name (used to reference it in routes)
• A base URL (API endpoint)
• An API key or credentials (for authentication)
• Optional: region (for services like Vertex AI)

Adding a Provider

1. Log into your Admin Bud-E dashboard
2. Navigate to Providers
3. Click Add Provider
4. Fill in the required fields
5. Click Save

Common Providers

Google Vertex AI

Use case: EU-compliant LLM/VLM/TTS/ASR with Gemini models

Configuration:

• Name: vertex (must be exact)
• Base URL: Leave empty (uses built-in proxy)
• API Key: Paste entire JSON key from service account
• Region: europe-west4 (or your chosen EU region)

TIP

See the Vertex AI setup guide for detailed instructions.

Together AI

Use case: Fast inference, wide model selection

Configuration:

• Name: together (or any name you prefer)
• Base URL: https://api.together.xyz
• API Key: Your Together API key (get it from together.ai)
• Region: (not applicable)

Example models:

• meta-llama/Llama-3-70b-chat-hf
• mistralai/Mixtral-8x7B-Instruct-v0.1

Mistral AI

Use case: European provider, strong multilingual models

Configuration:

• Name: mistral (or any name you prefer)
• Base URL: https://api.mistral.ai
• API Key: Your Mistral API key (get it from mistral.ai)
• Region: (not applicable)

Example models:

• mistral-large-latest
• mistral-medium-latest
• mistral-small-latest

OpenAI

Use case: GPT models (note: typically US-based, check their data policy)

Configuration:

• Name: openai (or any name you prefer)
• Base URL: https://api.openai.com
• API Key: Your OpenAI API key
• Region: (not applicable for most use cases)

EU Compliance

OpenAI typically processes data in US regions. Review their data usage policies if you need EU data residency.

Anthropic Claude (via Bedrock or direct)

Use case: High-quality reasoning, long context

Configuration (direct API):

• Name: anthropic (or any name you prefer)
• Base URL: https://api.anthropic.com
• API Key: Your Anthropic API key
• Region: (not applicable)

INFO

If using AWS Bedrock, configure with Bedrock endpoint and credentials instead.

Provider Name Requirements

The provider name you choose has two uses:

1. Reference in Routes: You'll select this name when creating routes
2. Built-in Proxies: Some names like vertex trigger special handling (OpenAI-compatible proxy)

Rules:

• Must be unique
• Case-sensitive (but lowercase recommended)
• No spaces or special characters
• Keep it short and memorable

Special names:

• vertex — Activates OpenAI-to-Vertex proxy
• (Other special names may be added in future versions)

API Key Security

Best Practices

1. Never share keys publicly

   • Don't commit to version control
   • Don't include in screenshots
   • Don't share in support requests

2. Rotate keys regularly

   • Set a reminder to rotate every 90 days
   • Rotate immediately if compromised

3. Use least-privilege access

   • Service accounts should have only required permissions
   • For Google Cloud: specific roles (not "Owner" or "Editor")

4. Monitor usage

   • Check provider dashboards for unexpected activity
   • Set up billing alerts
   • Review Admin Bud-E usage reports

Storing Keys

For JSON keys (like Google Cloud):

• Store in a password manager
• Encrypt backups
• Restrict file permissions on server: chmod 600 /path/to/key.json

For API tokens:

• Use environment variables on the server when possible
• Store in encrypted configuration files
• Never log the full key value

Testing a Provider

After adding a provider:

1. Create a test route pointing to it
2. Make a small test request from the frontend
3. Check the Usage page for:
   • Request appeared in logs
   • No error messages
   • Correct provider was used
   • Credits deducted accurately

Editing Providers

To update an existing provider:

1. Navigate to Providers
2. Click Edit on the provider you want to change
3. Update fields as needed
4. Click Save

WARNING

Changing a provider's name will break any routes that reference the old name. Update routes accordingly.

Deleting Providers

To remove a provider:

1. Navigate to Providers
2. Click Delete on the provider
3. Confirm deletion

DANGER

You cannot delete a provider that is currently referenced in active routes. Remove or update those routes first.

Failover and Redundancy

For high availability, add multiple providers for the same service type:

Example: LLM with failover

1. Provider 1: vertex (Priority 1)
2. Provider 2: together (Priority 2)
3. Provider 3: mistral (Priority 3)

If Vertex returns a 429 (rate limit) or 5xx (server error), Admin Bud-E automatically tries Together, then Mistral.

See Route Configuration for details on setting up failover.

EU Data Residency

For GDPR compliance, prefer providers with EU data processing:

Strong EU options:

• Google Vertex AI with EU regions (europe-west1/3/4/9)
• Mistral AI (French company, EU infrastructure)

Check documentation:

• Together AI: Review their data processing agreement
• OpenAI: Typically US-based (enterprise plans may offer EU routing)

Always review the provider's:

• Data Processing Agreement (DPA)
• Terms of Service
• Data residency options
• Sub-processor list

Cost Management

Setting Budget Alerts

Most providers let you set spending alerts:

Google Cloud:

1. Navigate to Billing → Budgets & alerts
2. Create a budget
3. Set alert thresholds (e.g., 50%, 90%, 100%)
4. Add email recipients

Together/Mistral/OpenAI:

• Check their dashboard for usage alerts
• Set up billing notifications
• Configure spending limits if available

Monitoring Costs

Use Admin Bud-E's Usage Reports to:

• Track per-provider spending
• Identify expensive models or users
• Forecast monthly costs
• Compare provider costs

See Usage Reports for details.

Next Steps

• Configure routes for failover
• Set up pricing per model
• Review privacy and EU compliance