Resilient Caching System

Overview

The BFAPI (ARCHIVED - migrate to LangChain ecosystem) environments where reliability, performance, and fault tolerance are critical. This document explains how to use and configure the resilient caching features to maximize stability and performance in your application.

Key Features

• Fallback Caching: Automatically use a secondary cache when the primary fails
• Automatic Retry: Configurable retry policy for transient errors
• Self-Healing: Automatic recovery from cache failures
• Cache Monitoring: Telemetry hooks for monitoring cache health and performance
• Error Handling: Graceful degradation when caching systems fail
• Timeout Protection: Prevent blocking operations when cache is slow or unresponsive
• Quota Management: Intelligent storage management for browser-based caches

Getting Started

Basic Setup

The simplest way to use resilient caching is through the CacheManager.createResilient() factory method:

import [CacheManager] from '@bluefly/bfapiclient';

// Create a resilient cache with default settings
const cacheManager = CacheManager.createResilient();

// Use like a regular cache
await cacheManager.set('key', 'value');
const result = await cacheManager.get('key');

By default, this creates:

• An in-memory primary cache
• An appropriate fallback cache (localStorage in browsers, in-memory in Node.js)
• A reasonable retry policy (2 retries with exponential backoff)

Custom Configuration

For more control, you can customize all aspects of the resilient cache:

import { CacheManager, InMemoryCache, LocalStorageCache, CacheEvent } from '@bluefly/bfapiclient';

// Custom telemetry function
function telemetry(event, details) {
  console.log(`Cache [event]: [details.operation]`, details);
  // In production, send to your monitoring system
}

// Create a resilient cache with custom configuration
const cacheManager = CacheManager.createResilient({
  // Primary and fallback caches
  primary: new InMemoryCache(),
  fallback: new LocalStorageCache('api_cache_'),

  // Retry policy
  retries: 3,
  retryDelayMs: 50,

  // Timeout protection
  timeoutMs: 2000,

  // Monitoring
  telemetry,

  // TTL settings
  defaultTtl: 600000, // 10 minutes
  fallbackTtl: primaryTtl => (primaryTtl || 600000) * 2, // Double TTL for fallback

  // Enable/disable
  enabled: true,
});

Cache Strategies

The resilient caching system supports multiple cache strategies that can be combined:

InMemoryCache

A fast, in-memory cache ideal for primary caching:

import [InMemoryCache] from '@bluefly/bfapiclient';

const memoryCache = new InMemoryCache();

LocalStorageCache

A persistent browser-based cache with enhanced reliability:

import [LocalStorageCache] from '@bluefly/bfapiclient';

// Optional prefix to namespace your cache entries
const storageCache = new LocalStorageCache('myapp_cache_');

The LocalStorageCache includes:

• Quota management to prevent storage limits from being exceeded
• Error handling for browser security restrictions
• Batch processing to avoid blocking the main thread
• Automatic cleanup of expired or corrupted entries

ResilientCache

A wrapper strategy that adds resilience features to any underlying cache:

import { ResilientCache, InMemoryCache } from '@bluefly/bfapiclient';

// Directly using ResilientCache (though createResilient() is preferred)
const resilientCache = new ResilientCache({
  primary: new InMemoryCache(),
  fallback: new LocalStorageCache(),
  retries: 2,
  telemetry: (event, details) => console.log(event, details),
});

Monitoring Cache Health

The resilient caching system provides telemetry for monitoring cache health:

import { CacheManager, CacheEvent } from '@bluefly/bfapiclient';

const cacheManager = CacheManager.createResilient({
  telemetry: (event, details) => {
    // Log or send to your monitoring system
    if (event === CacheEvent.ERROR) {
      console.error('Cache error:', details.error);
      // Alert or report the error
    }

    if (event === CacheEvent.FALLBACK_USED) {
      console.warn('Using fallback cache:', details);
      // Track fallback usage
    }

    // Track cache hit rate
    if (event === CacheEvent.HIT || event === CacheEvent.MISS) {
      trackCacheHitRate(event === CacheEvent.HIT);
    }
  },
});

Available cache events:

• CacheEvent.HIT: Cache hit in primary cache
• CacheEvent.MISS: Cache miss in all caches
• CacheEvent.ERROR: Error in cache operation
• CacheEvent.SET_SUCCESS: Successful set operation
• CacheEvent.SET_ERROR: Error in set operation
• CacheEvent.FALLBACK_USED: Fallback cache was used
• CacheEvent.RECOVERY_ATTEMPT: Attempting to recover primary cache
• CacheEvent.RECOVERY_SUCCESS: Primary cache recovered successfully
• CacheEvent.INVALIDATE: Cache entry invalidated
• CacheEvent.CLEAR: Cache cleared

Production Recommendations

For production environments, we recommend:

1. Use Multiple Cache Layers:

   • Fast in-memory cache for primary
   • Persistent storage for fallback (localStorage, IndexedDB)

2. Configure Appropriate Timeouts:

   • Short timeouts (1-2 seconds) for UI applications
   • Longer timeouts for background operations

3. Implement Comprehensive Telemetry:

   • Track cache hit rates
   • Monitor fallback usage
   • Alert on persistent errors

4. Tune TTL Settings:

   • Shorter TTL for volatile data
   • Longer TTL for fallback caches
   • Consider data freshness requirements

5. Error Handling:

   • Ensure your application can operate with degraded caching
   • Implement graceful fallbacks when all caches fail

Example Production Setup

import { CacheManager, InMemoryCache, LocalStorageCache } from '@bluefly/bfapiclient';

// Production-ready cache configuration
const cacheManager = CacheManager.createResilient({
  // Fast in-memory primary cache
  primary: new InMemoryCache(),

  // Persistent fallback for offline support
  fallback: new LocalStorageCache('prod_api_cache_'),

  // Aggressive retry for reliability
  retries: 3,
  retryDelayMs: 100,

  // Prevent blocking UI
  timeoutMs: 1000,

  // Production telemetry
  telemetry: productionTelemetryFunction,

  // Extended fallback TTL for better offline experience
  fallbackTtl: primaryTtl => Math.min(primaryTtl * 3, 24 * 60 * 60 * 1000),

  // Default TTL of 10 minutes
  defaultTtl: 600000,
});

Best Practices

1. Cache Invalidation:

   • Implement consistent cache invalidation strategies
   • Use short TTLs for frequently changing data
   • Invalidate related entries together

2. Storage Optimization:

   • Monitor cache size in browser environments
   • Implement LRU or similar eviction policies for large caches
   • Be mindful of localStorage limits (typically 5-10MB)

3. Testing:

   • Test cache behavior with network errors
   • Verify fallback behavior works correctly
   • Include cache failures in integration tests

4. Security Considerations:

   • Never cache sensitive information in browser storage
   • Consider cache poisoning in security threat models
   • Implement appropriate cache headers for API responses

---

By leveraging the resilient caching system, your application can maintain high performance and reliability even when facing transient network issues, browser storage limitations, or other common failure modes.