Skip to content

Deployment Guide

This guide provides instructions for deploying the Bluefly LLM ecosystem to various environments.

Deployment Environments

The Bluefly LLM ecosystem supports the following deployment environments:

Development

  • Local environment for developers
  • Minimal resource requirements
  • Configured through .env files

Staging

  • Pre-production environment for testing
  • Mirrors production configuration
  • Accessible to internal teams only

Production

  • Live environment for end users
  • High availability and scalability
  • Strict security controls

Deployment Architecture

The standard deployment architecture includes:

  • API Services: BFAPI instances behind load balancers
  • ML Services: BFLLM instances for model inference and training
  • Databases: MongoDB for structured data storage
  • Cache: Redis for caching frequently accessed data
  • Storage: S3-compatible object storage for models and artifacts
  • UI: Static assets served through CDN

Deployment Methods

Docker Deployment

  1. Build Docker images for each component:
# Build API Docker image
cd bfapi
docker build -t bluefly/bfapi:latest .

# Build BFLLM Docker image
cd ../bfllm
docker build -t bluefly/bfllm:latest .
  1. Deploy using Docker Compose:
docker-compose -f docker-compose.yml up -d

Kubernetes Deployment

  1. Apply Kubernetes manifests:
kubectl apply -f kubernetes/namespace.yaml
kubectl apply -f kubernetes/configmap.yaml
kubectl apply -f kubernetes/secrets.yaml
kubectl apply -f kubernetes/deployments/
kubectl apply -f kubernetes/services/
kubectl apply -f kubernetes/ingress.yaml
  1. Verify deployment:
kubectl get pods -n bluefly
kubectl get services -n bluefly

Cloud Provider Deployment

AWS

  1. Set up infrastructure using Terraform:
cd terraform/aws
terraform init
terraform apply
  1. Deploy services using AWS ECS or EKS.

Google Cloud

  1. Set up infrastructure using Terraform:
cd terraform/gcp
terraform init
terraform apply
  1. Deploy services using Google Kubernetes Engine.

Configuration Management

Environment Variables

Each component requires specific environment variables. These can be provided through:

  • .env files for local development
  • ConfigMaps and Secrets for Kubernetes
  • Environment configuration in CI/CD pipelines

Secrets Management

Sensitive configuration such as API keys and credentials should be managed using:

  • GitLab CI/CD Variables for pipeline secrets
  • Kubernetes Secrets for deployment secrets
  • AWS Secrets Manager or Google Secret Manager for cloud deployments

Deployment Validation

After deployment, validate the system using:

  1. Health check endpoints
  2. Integration tests against deployed services
  3. End-to-end workflow validation

Monitoring and Logging

Ensure monitoring and logging are configured:

  • Prometheus for metrics collection
  • Grafana for metrics visualization
  • ELK stack for log aggregation and analysis
  • Alert rules for critical service metrics

Rollback Procedures

In case of deployment issues:

  1. Identify the problematic component or change
  2. Revert to the previous stable version
  3. Restore from backup if data corruption occurred
  4. Conduct post-mortem analysis

Continuous Deployment

The Bluefly LLM ecosystem supports continuous deployment through:

  • GitLab CI/CD pipelines
  • Automated tests in deployment pipeline
  • Progressive deployment strategies (blue/green, canary)

Maintenance Windows

Scheduled maintenance should be conducted during low-usage periods:

  • Database maintenance: Weekends, 2-4 AM UTC
  • Service updates: Weekdays, 3-5 AM UTC
  • Notify users 48 hours in advance for planned maintenance