--- url: /guide/implementations/aws.md --- # AWS Implementation Foundatio provides AWS implementations for file storage and queuing using Amazon S3 and Amazon SQS. ## Overview | Implementation | Interface | Package | |----------------|-----------|---------| | `S3FileStorage` | `IFileStorage` | Foundatio.AWS | | `SQSQueue` | `IQueue` | Foundatio.AWS | ## Installation ```bash dotnet add package Foundatio.AWS ``` ## Amazon S3 Storage Store files in Amazon S3 with full support for buckets, prefixes, and metadata. ### Basic Usage ```csharp using Foundatio.Storage; using Amazon; var storage = new S3FileStorage(options => { options.Bucket = "my-files"; options.Region = RegionEndpoint.USEast1; }); // Save a file await storage.SaveFileAsync("documents/report.pdf", pdfStream); // Read a file var stream = await storage.GetFileStreamAsync("documents/report.pdf"); // Get file contents as string var content = await storage.GetFileContentsAsync("config/settings.json"); ``` ### Configuration Options ```csharp var storage = new S3FileStorage(options => { // Bucket name options.Bucket = "my-files"; // AWS Region options.Region = RegionEndpoint.USEast1; // Explicit credentials (optional - uses default chain if not specified) options.AccessKey = "AKIAIOSFODNN7EXAMPLE"; options.SecretKey = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"; // Service URL for localstack or custom endpoints options.ServiceUrl = "http://localhost:4566"; // Logger options.LoggerFactory = loggerFactory; // Serializer for metadata options.Serializer = serializer; }); ``` ### Using IAM Roles (Recommended) ```csharp // When running on AWS (EC2, ECS, Lambda, etc.) // credentials are automatically picked up from IAM role var storage = new S3FileStorage(options => { options.Bucket = "my-files"; options.Region = RegionEndpoint.USEast1; // No credentials specified - uses instance profile }); ``` ### Using AWS Profiles ```csharp using Amazon.Runtime.CredentialManagement; var chain = new CredentialProfileStoreChain(); chain.TryGetAWSCredentials("my-profile", out var credentials); var storage = new S3FileStorage(options => { options.Bucket = "my-files"; options.Region = RegionEndpoint.USEast1; options.Credentials = credentials; }); ``` ### File Operations ```csharp // List files with prefix var files = await storage.GetFileListAsync("documents/"); await foreach (var file in files) { Console.WriteLine($"{file.Path} - {file.Size} bytes"); } // Check existence if (await storage.ExistsAsync("documents/report.pdf")) { // File exists } // Get file info var info = await storage.GetFileInfoAsync("documents/report.pdf"); Console.WriteLine($"Size: {info?.Size}, Modified: {info?.Modified}"); // Copy files await storage.CopyFileAsync("source.txt", "backup/source.txt"); // Delete files await storage.DeleteFileAsync("old-file.txt"); // Delete multiple files await storage.DeleteFilesAsync("temp/"); // Delete by prefix ``` ### Presigned URLs ```csharp // Generate presigned URL for direct client access var client = new AmazonS3Client(); var request = new GetPreSignedUrlRequest { BucketName = "my-files", Key = "documents/report.pdf", Expires = DateTime.UtcNow.AddHours(1) }; var url = client.GetPreSignedURL(request); ``` ### DI Registration ```csharp services.AddSingleton(sp => new S3FileStorage(options => { options.Bucket = configuration["AWS:S3:Bucket"]; options.Region = RegionEndpoint.GetBySystemName( configuration["AWS:Region"] ?? "us-east-1"); options.LoggerFactory = sp.GetRequiredService(); })); ``` ## Amazon SQS Queue Use Amazon Simple Queue Service for reliable message queuing. ### Basic Usage ```csharp using Foundatio.Queues; using Amazon; var queue = new SQSQueue(options => { options.Name = "work-items"; options.Region = RegionEndpoint.USEast1; }); // Enqueue await queue.EnqueueAsync(new WorkItem { Id = 1, Data = "Process this" }); // Dequeue var entry = await queue.DequeueAsync(); if (entry != null) { await ProcessAsync(entry.Value); await entry.CompleteAsync(); } ``` ### Configuration Options ```csharp var queue = new SQSQueue(options => { // Queue name options.Name = "work-items"; // AWS Region options.Region = RegionEndpoint.USEast1; // Optional credentials options.AccessKey = "..."; options.SecretKey = "..."; // Visibility timeout options.WorkItemTimeout = TimeSpan.FromMinutes(5); // Retry settings options.Retries = 3; options.RetryDelay = TimeSpan.FromSeconds(30); // Long polling (reduces costs) options.WaitTimeSeconds = 20; // Batch receive options.MaxNumberOfMessages = 10; // Auto-create queue options.AutoCreateQueue = true; options.LoggerFactory = loggerFactory; }); ``` ### FIFO Queues ```csharp // FIFO queues guarantee message order and exactly-once delivery var queue = new SQSQueue(options => { options.Name = "orders.fifo"; // Must end with .fifo options.Region = RegionEndpoint.USEast1; options.IsFifo = true; }); // Enqueue with message group await queue.EnqueueAsync(item, new QueueEntryOptions { Properties = new Dictionary { ["MessageGroupId"] = orderId, ["MessageDeduplicationId"] = Guid.NewGuid().ToString() } }); ``` ### Dead Letter Queue ```csharp // Configure dead letter queue in AWS Console or via CloudFormation // Foundatio will respect the DLQ configuration var queue = new SQSQueue(options => { options.Name = "work-items"; options.Retries = 3; // After 3 failures, moves to DLQ }); ``` ### Processing Patterns ```csharp // Continuous processing await queue.StartWorkingAsync(async (entry, token) => { await ProcessWorkItemAsync(entry.Value); }); // Batch processing var entries = new List>(); while (true) { var entry = await queue.DequeueAsync(TimeSpan.FromSeconds(1)); if (entry == null) break; entries.Add(entry); } // Process batch foreach (var entry in entries) { await ProcessAsync(entry.Value); await entry.CompleteAsync(); } ``` ### DI Registration ```csharp services.AddSingleton>(sp => new SQSQueue(options => { options.Name = configuration["AWS:SQS:QueueName"]; options.Region = RegionEndpoint.GetBySystemName( configuration["AWS:Region"] ?? "us-east-1"); options.LoggerFactory = sp.GetRequiredService(); })); ``` ## Complete AWS Setup ### Combined Services ```csharp public static IServiceCollection AddFoundatioAWS( this IServiceCollection services, IConfiguration configuration) { var region = RegionEndpoint.GetBySystemName( configuration["AWS:Region"] ?? "us-east-1"); // S3 Storage services.AddSingleton(sp => new S3FileStorage(options => { options.Bucket = configuration["AWS:S3:Bucket"]; options.Region = region; options.LoggerFactory = sp.GetRequiredService(); })); return services; } // Add SQS queue public static IServiceCollection AddSQSQueue( this IServiceCollection services, string name, IConfiguration configuration) where T : class { var region = RegionEndpoint.GetBySystemName( configuration["AWS:Region"] ?? "us-east-1"); services.AddSingleton>(sp => new SQSQueue(options => { options.Name = name; options.Region = region; options.LoggerFactory = sp.GetRequiredService(); })); return services; } ``` ### Configuration ```json { "AWS": { "Region": "us-east-1", "S3": { "Bucket": "my-app-files" }, "SQS": { "QueueName": "work-items" } } } ``` ### Environment Variables ```bash # Standard AWS environment variables AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY AWS_REGION=us-east-1 AWS_PROFILE=my-profile # Application-specific AWS_S3_BUCKET=my-app-files AWS_SQS_QUEUE_NAME=work-items ``` ## Local Development with LocalStack ### Docker Compose ```yaml version: '3.8' services: localstack: image: localstack/localstack ports: - "4566:4566" environment: - SERVICES=s3,sqs - DEBUG=1 - DATA_DIR=/tmp/localstack/data volumes: - "./localstack:/tmp/localstack" ``` ### Configuration for LocalStack ```csharp var storage = new S3FileStorage(options => { options.Bucket = "test-bucket"; options.ServiceUrl = "http://localhost:4566"; options.ForcePathStyle = true; // Required for LocalStack options.AccessKey = "test"; options.SecretKey = "test"; }); var queue = new SQSQueue(options => { options.Name = "test-queue"; options.ServiceUrl = "http://localhost:4566"; options.AccessKey = "test"; options.SecretKey = "test"; }); ``` ### Create Resources ```bash # Create S3 bucket aws --endpoint-url=http://localhost:4566 s3 mb s3://test-bucket # Create SQS queue aws --endpoint-url=http://localhost:4566 sqs create-queue --queue-name test-queue ``` ## Production Considerations ### IAM Policies ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::my-files", "arn:aws:s3:::my-files/*" ] }, { "Effect": "Allow", "Action": [ "sqs:SendMessage", "sqs:ReceiveMessage", "sqs:DeleteMessage", "sqs:GetQueueAttributes" ], "Resource": "arn:aws:sqs:us-east-1:*:work-items" } ] } ``` ### Health Checks ```csharp builder.Services.AddHealthChecks() .AddS3(options => { options.BucketName = "my-files"; options.S3Config = new AmazonS3Config { RegionEndpoint = RegionEndpoint.USEast1 }; }, name: "aws-s3") .AddSqs(options => { options.QueueUrl = "https://sqs.us-east-1.amazonaws.com/123456789/work-items"; options.Config = new AmazonSQSConfig { RegionEndpoint = RegionEndpoint.USEast1 }; }, name: "aws-sqs"); ``` ### Cost Optimization ```csharp // S3: Use appropriate storage classes var putRequest = new PutObjectRequest { BucketName = "my-files", Key = "archive/old-data.zip", InputStream = stream, StorageClass = S3StorageClass.IntelligentTiering }; // SQS: Use long polling to reduce costs var queue = new SQSQueue(options => { options.WaitTimeSeconds = 20; // Long poll for 20 seconds }); // SQS: Batch operations var queue = new SQSQueue(options => { options.MaxNumberOfMessages = 10; // Receive up to 10 at once }); ``` ### Encryption ```csharp // S3: Server-side encryption var storage = new S3FileStorage(options => { options.Bucket = "my-files"; options.ServerSideEncryption = ServerSideEncryptionMethod.AES256; // Or use KMS options.ServerSideEncryption = ServerSideEncryptionMethod.AWSKMS; options.ServerSideEncryptionKeyManagementServiceKeyId = "key-id"; }); // SQS: Enable encryption in AWS Console or CloudFormation ``` ## Best Practices ### 1. Use IAM Roles ```csharp // ✅ IAM Role (no credentials in code) var storage = new S3FileStorage(options => { options.Bucket = "my-files"; options.Region = RegionEndpoint.USEast1; }); // ❌ Hardcoded credentials options.AccessKey = "AKIA..."; options.SecretKey = "..."; ``` ### 2. Bucket Naming ```csharp // ✅ Globally unique, lowercase options.Bucket = "mycompany-app-files-prod"; // ❌ Invalid names options.Bucket = "MyBucket"; // No uppercase options.Bucket = "files"; // Too generic, may conflict ``` ### 3. Handle Transient Failures ```csharp // AWS SDK has built-in retry // Add application-level resilience for business logic var policy = new ResiliencePolicy { MaxAttempts = 3, GetDelay = ResiliencePolicy.ExponentialDelay(TimeSpan.FromSeconds(1)) }; await policy.ExecuteAsync(async ct => { await storage.SaveFileAsync("file.txt", content); }, cancellationToken); ``` ### 4. Use Appropriate Timeouts ```csharp // For large file operations var config = new AmazonS3Config { RegionEndpoint = RegionEndpoint.USEast1, Timeout = TimeSpan.FromMinutes(5), ReadWriteTimeout = TimeSpan.FromMinutes(5) }; ``` ### 5. Enable Logging ```csharp // AWS SDK logging AWSConfigs.LoggingConfig.LogTo = LoggingOptions.Console; AWSConfigs.LoggingConfig.LogResponses = ResponseLoggingOption.OnError; ``` ## S3 vs SQS Features ### S3 Storage Features * Object storage (files of any size) * Versioning * Lifecycle policies * Cross-region replication * Event notifications * Static website hosting ### SQS Queue Features * At-least-once delivery * FIFO queues (exactly-once) * Dead letter queues * Long polling * Message delay * Batch operations ## Next Steps * [Azure Implementation](./azure) - Azure Storage and Service Bus * [Redis Implementation](./redis) - Distributed caching * [In-Memory Implementation](./in-memory) - Local development --- --- url: /guide/implementations/azure.md --- # Azure Implementation Foundatio provides Azure implementations for storage and messaging using Azure Blob Storage and Azure Service Bus. ## Overview | Implementation | Interface | Package | |----------------|-----------|---------| | `AzureFileStorage` | `IFileStorage` | Foundatio.AzureStorage | | `AzureStorageQueue` | `IQueue` | Foundatio.AzureStorage | | `AzureServiceBusQueue` | `IQueue` | Foundatio.AzureServiceBus | | `AzureServiceBusMessageBus` | `IMessageBus` | Foundatio.AzureServiceBus | ## Installation ```bash # Azure Storage (Blobs and Queues) dotnet add package Foundatio.AzureStorage # Azure Service Bus (Queues and Messaging) dotnet add package Foundatio.AzureServiceBus ``` ## Azure Blob Storage Store files in Azure Blob Storage with full support for containers, virtual directories, and metadata. ### Basic Usage ```csharp using Foundatio.Storage; var storage = new AzureFileStorage(options => { options.ConnectionString = "DefaultEndpointsProtocol=https;AccountName=..."; options.ContainerName = "files"; }); // Save a file await storage.SaveFileAsync("documents/report.pdf", pdfStream); // Read a file var stream = await storage.GetFileStreamAsync("documents/report.pdf"); // Get file contents as string var content = await storage.GetFileContentsAsync("config/settings.json"); ``` ### Configuration Options ```csharp var storage = new AzureFileStorage(options => { // Connection string options.ConnectionString = connectionString; // Container name options.ContainerName = "myfiles"; // Create container if not exists options.ShouldCreateContainer = true; // Logger options.LoggerFactory = loggerFactory; // Serializer for metadata options.Serializer = serializer; }); ``` ### Using Managed Identity ```csharp using Azure.Identity; var storage = new AzureFileStorage(options => { options.BlobServiceClient = new BlobServiceClient( new Uri("https://mystorageaccount.blob.core.windows.net"), new DefaultAzureCredential() ); options.ContainerName = "files"; }); ``` ### File Operations ```csharp // List files var files = await storage.GetFileListAsync("documents/"); await foreach (var file in files) { Console.WriteLine($"{file.Path} - {file.Size} bytes"); } // Check existence if (await storage.ExistsAsync("documents/report.pdf")) { // File exists } // Get file info var info = await storage.GetFileInfoAsync("documents/report.pdf"); Console.WriteLine($"Modified: {info?.Modified}"); // Copy files await storage.CopyFileAsync("source.txt", "backup/source.txt"); // Delete files await storage.DeleteFileAsync("old-file.txt"); await storage.DeleteFilesAsync("temp/"); // Delete folder ``` ### DI Registration ```csharp services.AddSingleton(sp => new AzureFileStorage(options => { options.ConnectionString = configuration.GetConnectionString("AzureStorage"); options.ContainerName = "files"; options.LoggerFactory = sp.GetRequiredService(); })); ``` ## Azure Storage Queue Use Azure Storage Queues for simple, reliable message queuing. ### Basic Usage ```csharp using Foundatio.Queues; var queue = new AzureStorageQueue(options => { options.ConnectionString = connectionString; options.Name = "work-items"; }); // Enqueue await queue.EnqueueAsync(new WorkItem { Id = 1, Data = "Process this" }); // Dequeue var entry = await queue.DequeueAsync(); if (entry != null) { await ProcessAsync(entry.Value); await entry.CompleteAsync(); } ``` ### Configuration Options ```csharp var queue = new AzureStorageQueue(options => { options.ConnectionString = connectionString; options.Name = "work-items"; // Visibility timeout options.WorkItemTimeout = TimeSpan.FromMinutes(5); // Retry settings options.Retries = 3; options.RetryDelay = TimeSpan.FromSeconds(30); // Dequeue batch size options.DequeueCount = 1; options.LoggerFactory = loggerFactory; }); ``` ### DI Registration ```csharp services.AddSingleton>(sp => new AzureStorageQueue(options => { options.ConnectionString = configuration.GetConnectionString("AzureStorage"); options.Name = "work-items"; options.LoggerFactory = sp.GetRequiredService(); })); ``` ## Azure Service Bus Queue Use Azure Service Bus for enterprise-grade messaging with advanced features. ### Basic Usage ```csharp using Foundatio.Queues; var queue = new AzureServiceBusQueue(options => { options.ConnectionString = serviceBusConnectionString; options.Name = "work-items"; }); // Enqueue await queue.EnqueueAsync(new WorkItem { Id = 1 }); // Process messages await queue.StartWorkingAsync(async (entry, token) => { await ProcessWorkItemAsync(entry.Value); }); ``` ### Configuration Options ```csharp var queue = new AzureServiceBusQueue(options => { options.ConnectionString = connectionString; options.Name = "work-items"; // Processing options options.WorkItemTimeout = TimeSpan.FromMinutes(5); options.Retries = 3; options.AutoCreateQueue = true; // Prefetch for better performance options.PrefetchCount = 10; options.LoggerFactory = loggerFactory; }); ``` ### Advanced Features ```csharp // Scheduled messages await queue.EnqueueAsync(new WorkItem { Id = 1 }, new QueueEntryOptions { DeliveryDelay = TimeSpan.FromHours(1) }); // Sessions (ordered processing) var queue = new AzureServiceBusQueue(options => { options.ConnectionString = connectionString; options.Name = "orders"; options.RequiresSession = true; }); // Enqueue with session await queue.EnqueueAsync(item, new QueueEntryOptions { Properties = new Dictionary { ["SessionId"] = orderId } }); ``` ### DI Registration ```csharp services.AddSingleton>(sp => new AzureServiceBusQueue(options => { options.ConnectionString = configuration.GetConnectionString("ServiceBus"); options.Name = "work-items"; options.LoggerFactory = sp.GetRequiredService(); })); ``` ## Azure Service Bus Message Bus Use Azure Service Bus Topics for pub/sub messaging. ### Basic Usage ```csharp using Foundatio.Messaging; var messageBus = new AzureServiceBusMessageBus(options => { options.ConnectionString = connectionString; options.Topic = "events"; }); // Subscribe await messageBus.SubscribeAsync(async message => { await HandleOrderCreatedAsync(message); }); // Publish await messageBus.PublishAsync(new OrderCreatedEvent { OrderId = "123" }); ``` ### Configuration Options ```csharp var messageBus = new AzureServiceBusMessageBus(options => { options.ConnectionString = connectionString; options.Topic = "events"; // Subscription name (unique per consumer) options.SubscriptionName = "order-processor"; // Auto-create topic/subscription options.AutoCreateTopic = true; // Message prefetch options.PrefetchCount = 10; options.LoggerFactory = loggerFactory; }); ``` ### Multiple Subscribers ```csharp // Each service gets its own subscription // All subscribers receive all messages // Order Service var orderBus = new AzureServiceBusMessageBus(options => { options.Topic = "events"; options.SubscriptionName = "order-service"; }); // Notification Service var notificationBus = new AzureServiceBusMessageBus(options => { options.Topic = "events"; options.SubscriptionName = "notification-service"; }); // Both receive OrderCreatedEvent await messageBus.PublishAsync(new OrderCreatedEvent()); ``` ### DI Registration ```csharp services.AddSingleton(sp => new AzureServiceBusMessageBus(options => { options.ConnectionString = configuration.GetConnectionString("ServiceBus"); options.Topic = "events"; options.SubscriptionName = configuration["ServiceBus:SubscriptionName"]; options.LoggerFactory = sp.GetRequiredService(); })); services.AddSingleton(sp => sp.GetRequiredService()); services.AddSingleton(sp => sp.GetRequiredService()); ``` ## Complete Azure Setup ### Combined Services ```csharp public static IServiceCollection AddFoundatioAzure( this IServiceCollection services, IConfiguration configuration) { var storageConnection = configuration.GetConnectionString("AzureStorage"); var serviceBusConnection = configuration.GetConnectionString("ServiceBus"); // File Storage services.AddSingleton(sp => new AzureFileStorage(options => { options.ConnectionString = storageConnection; options.ContainerName = "files"; options.LoggerFactory = sp.GetRequiredService(); })); // Message Bus services.AddSingleton(sp => new AzureServiceBusMessageBus(options => { options.ConnectionString = serviceBusConnection; options.Topic = "events"; options.SubscriptionName = configuration["Azure:ServiceBus:SubscriptionName"]; options.LoggerFactory = sp.GetRequiredService(); })); services.AddSingleton(sp => sp.GetRequiredService()); services.AddSingleton(sp => sp.GetRequiredService()); return services; } // Add Service Bus queue public static IServiceCollection AddAzureServiceBusQueue( this IServiceCollection services, string name, IConfiguration configuration) where T : class { services.AddSingleton>(sp => new AzureServiceBusQueue(options => { options.ConnectionString = configuration.GetConnectionString("ServiceBus"); options.Name = name; options.LoggerFactory = sp.GetRequiredService(); })); return services; } ``` ### Configuration ```json { "ConnectionStrings": { "AzureStorage": "DefaultEndpointsProtocol=https;AccountName=...;AccountKey=...;EndpointSuffix=core.windows.net", "ServiceBus": "Endpoint=sb://....servicebus.windows.net/;SharedAccessKeyName=...;SharedAccessKey=..." }, "Azure": { "ServiceBus": { "SubscriptionName": "my-service" } } } ``` ## Managed Identity ### Azure Storage with Managed Identity ```csharp using Azure.Identity; var storage = new AzureFileStorage(options => { options.BlobServiceClient = new BlobServiceClient( new Uri("https://mystorageaccount.blob.core.windows.net"), new DefaultAzureCredential() ); options.ContainerName = "files"; }); ``` ### Azure Service Bus with Managed Identity ```csharp using Azure.Identity; var messageBus = new AzureServiceBusMessageBus(options => { options.ServiceBusClient = new ServiceBusClient( "mynamespace.servicebus.windows.net", new DefaultAzureCredential() ); options.Topic = "events"; }); ``` ## Production Considerations ### Health Checks ```csharp builder.Services.AddHealthChecks() .AddAzureBlobStorage(storageConnection, name: "azure-storage") .AddAzureServiceBusQueue(serviceBusConnection, "work-items", name: "azure-servicebus-queue") .AddAzureServiceBusTopic(serviceBusConnection, "events", name: "azure-servicebus-topic"); ``` ### Retry Policies ```csharp // Azure SDK has built-in retry policies // Foundatio integrates with them automatically var storage = new AzureFileStorage(options => { options.BlobServiceClient = new BlobServiceClient( connectionString, new BlobClientOptions { Retry = { MaxRetries = 5, Delay = TimeSpan.FromSeconds(1), MaxDelay = TimeSpan.FromSeconds(30), Mode = RetryMode.Exponential } }); }); ``` ### Cost Optimization ```csharp // Use cool or archive tier for infrequently accessed files var blobClient = containerClient.GetBlobClient("archive/old-data.zip"); await blobClient.SetAccessTierAsync(AccessTier.Cool); // Batch operations where possible var files = new[] { "file1.txt", "file2.txt", "file3.txt" }; foreach (var file in files) { await storage.SaveFileAsync($"batch/{file}", content); } ``` ## Azure Storage vs Service Bus ### When to Use Azure Storage Queue * Simple message queuing * High volume, low latency not critical * Cost-sensitive scenarios * No advanced features needed ### When to Use Azure Service Bus * Enterprise messaging requirements * Ordered message processing (sessions) * Scheduled messages * Dead letter handling * Topics and subscriptions (pub/sub) * Transactions ## Best Practices ### 1. Use Managed Identity ```csharp // ✅ Managed Identity (no secrets in code) new DefaultAzureCredential() // ❌ Connection strings with secrets "AccountKey=..." ``` ### 2. Container Naming ```csharp // ✅ Lowercase, descriptive names options.ContainerName = "user-uploads"; // ❌ Invalid characters options.ContainerName = "User_Uploads"; // Invalid ``` ### 3. Handle Transient Failures ```csharp // Azure SDK handles retries automatically // Add application-level resilience for business logic var policy = new ResiliencePolicy { MaxAttempts = 3, GetDelay = ResiliencePolicy.ExponentialDelay(TimeSpan.FromSeconds(1)) }; await policy.ExecuteAsync(async ct => { await storage.SaveFileAsync("file.txt", content); }, cancellationToken); ``` ### 4. Monitor and Alert ```csharp // Use Application Insights builder.Services.AddApplicationInsightsTelemetry(); // Log operations logger.LogInformation("Saved file {Path} to Azure Storage", path); ``` ## Next Steps * [AWS Implementation](./aws) - S3 and SQS integration * [Redis Implementation](./redis) - Distributed caching * [In-Memory Implementation](./in-memory) - Local development --- --- url: /guide/caching.md --- # Caching Caching allows you to store and access data lightning fast, saving expensive operations to create or get data. Foundatio provides multiple cache implementations through the `ICacheClient` interface. ## The ICacheClient Interface ```csharp public interface ICacheClient : IDisposable { Task RemoveAsync(string key); Task RemoveIfEqualAsync(string key, T expected); Task RemoveAllAsync(IEnumerable keys = null); Task RemoveByPrefixAsync(string prefix); Task> GetAsync(string key); Task>> GetAllAsync(IEnumerable keys); Task AddAsync(string key, T value, TimeSpan? expiresIn = null); Task SetAsync(string key, T value, TimeSpan? expiresIn = null); Task SetAllAsync(IDictionary values, TimeSpan? expiresIn = null); Task ReplaceAsync(string key, T value, TimeSpan? expiresIn = null); Task ReplaceIfEqualAsync(string key, T value, T expected, TimeSpan? expiresIn = null); Task IncrementAsync(string key, double amount, TimeSpan? expiresIn = null); Task IncrementAsync(string key, long amount, TimeSpan? expiresIn = null); Task ExistsAsync(string key); Task GetExpirationAsync(string key); Task SetExpirationAsync(string key, TimeSpan expiresIn); Task SetIfHigherAsync(string key, double value, TimeSpan? expiresIn = null); Task SetIfHigherAsync(string key, long value, TimeSpan? expiresIn = null); Task SetIfLowerAsync(string key, double value, TimeSpan? expiresIn = null); Task SetIfLowerAsync(string key, long value, TimeSpan? expiresIn = null); Task ListAddAsync(string key, IEnumerable values, TimeSpan? expiresIn = null); Task ListRemoveAsync(string key, IEnumerable values, TimeSpan? expiresIn = null); Task>> GetListAsync(string key, int? page = null, int pageSize = 100); } ``` ## Implementations ### InMemoryCacheClient An in-memory cache implementation valid for the lifetime of the process: ```csharp using Foundatio.Caching; var cache = new InMemoryCacheClient(); // Basic operations await cache.SetAsync("key", "value"); var result = await cache.GetAsync("key"); // With expiration await cache.SetAsync("session", sessionData, TimeSpan.FromMinutes(30)); ``` #### MaxItems Configuration Limit the number of cached items (LRU eviction): ```csharp var cache = new InMemoryCacheClient(o => o.MaxItems = 250); // Only keeps the last 250 items accessed // Useful for caching resolved data like geo-ip lookups ``` ### HybridCacheClient Combines local in-memory caching with a distributed cache for maximum performance: ```csharp using Foundatio.Caching; var hybridCache = new HybridCacheClient( distributedCache: redisCacheClient, messageBus: redisMessageBus ); // First access: fetches from Redis, caches locally var user = await hybridCache.GetAsync("user:123"); // Subsequent access: returns from local cache (no network call) var sameUser = await hybridCache.GetAsync("user:123"); ``` **How it works:** 1. Reads check local cache first 2. On miss, reads from distributed cache and caches locally 3. Writes go to distributed cache and publish invalidation message 4. All instances receive invalidation and clear local cache **Benefits:** * **Huge performance gains**: Skip serialization and network calls * **Consistency**: Message bus keeps all instances in sync * **Automatic**: No manual cache invalidation logic ### ScopedCacheClient Prefix all cache keys for easy namespacing: ```csharp using Foundatio.Caching; var cache = new InMemoryCacheClient(); var tenantCache = new ScopedCacheClient(cache, "tenant:abc"); // All keys automatically prefixed await tenantCache.SetAsync("settings", settings); // Key: "tenant:abc:settings" await tenantCache.SetAsync("users", users); // Key: "tenant:abc:users" // Clear all keys for this tenant await tenantCache.RemoveByPrefixAsync(""); // Removes tenant:abc:* ``` **Use cases:** * Multi-tenant applications * Feature-specific caches * Test isolation ### RedisCacheClient Distributed cache using Redis (separate package): ```csharp // dotnet add package Foundatio.Redis using Foundatio.Redis.Cache; using StackExchange.Redis; var redis = await ConnectionMultiplexer.ConnectAsync("localhost:6379"); var cache = new RedisCacheClient(o => o.ConnectionMultiplexer = redis); await cache.SetAsync("user:123", user, TimeSpan.FromHours(1)); ``` ### RedisHybridCacheClient Combines `RedisCacheClient` with `HybridCacheClient`: ```csharp var redis = await ConnectionMultiplexer.ConnectAsync("localhost:6379"); var hybridCache = new RedisHybridCacheClient(o => { o.ConnectionMultiplexer = redis; o.LocalCacheMaxItems = 1000; }); ``` ## Common Patterns ### Cache-Aside Pattern The most common caching pattern: ```csharp public async Task GetUserAsync(int userId) { var cacheKey = $"user:{userId}"; // Try cache first var cached = await _cache.GetAsync(cacheKey); if (cached.HasValue) return cached.Value; // Load from database var user = await _database.GetUserAsync(userId); // Cache for future requests await _cache.SetAsync(cacheKey, user, TimeSpan.FromMinutes(30)); return user; } ``` ### Atomic Operations Use conditional operations for race-safe updates: ```csharp // Only set if key doesn't exist bool added = await cache.AddAsync("lock:resource", "owner-id"); // Replace only if value matches expected bool replaced = await cache.ReplaceIfEqualAsync("counter", 2, 1); // Atomic increment long newValue = await cache.IncrementAsync("page-views", 1); ``` ### Counter Patterns Track metrics with atomic operations: ```csharp // Increment counters await cache.IncrementAsync("api:calls:today", 1); await cache.IncrementAsync("user:123:login-count", 1); // Track high-water marks await cache.SetIfHigherAsync("max-concurrent-users", currentUsers); // Track minimums await cache.SetIfLowerAsync("fastest-response-ms", responseTime); ``` ### List Operations Store and manage lists: ```csharp // Add to a list await cache.ListAddAsync("user:123:recent-searches", new[] { "query1" }); // Get paginated list var searches = await cache.GetListAsync( "user:123:recent-searches", page: 0, pageSize: 10 ); // Remove from list await cache.ListRemoveAsync("user:123:recent-searches", new[] { "query1" }); ``` ### Bulk Operations Efficiently work with multiple keys: ```csharp // Get multiple values var keys = new[] { "user:1", "user:2", "user:3" }; var users = await cache.GetAllAsync(keys); // Set multiple values var values = new Dictionary { ["user:1"] = user1, ["user:2"] = user2, }; await cache.SetAllAsync(values, TimeSpan.FromHours(1)); // Remove multiple keys await cache.RemoveAllAsync(keys); ``` ## Dependency Injection ### Basic Registration ```csharp // In-memory (development) services.AddSingleton(); // With options services.AddSingleton(sp => new InMemoryCacheClient(o => o.MaxItems = 1000)); // Redis (production) services.AddSingleton(sp => new RedisCacheClient(o => o.ConnectionMultiplexer = redis)); ``` ### Hybrid with DI ```csharp services.AddSingleton(sp => { var redis = sp.GetRequiredService(); return new HybridCacheClient( new RedisCacheClient(o => o.ConnectionMultiplexer = redis), sp.GetRequiredService() ); }); ``` ### Named Caches Use different caches for different purposes: ```csharp services.AddKeyedSingleton("session", new InMemoryCacheClient(o => o.MaxItems = 10000)); services.AddKeyedSingleton("geo", new InMemoryCacheClient(o => o.MaxItems = 250)); ``` ## Best Practices ### 1. Use Meaningful Key Patterns ```csharp // ✅ Good: Clear, hierarchical, identifiable "user:123:profile" "tenant:abc:settings" "api:rate-limit:192.168.1.1" // ❌ Bad: Ambiguous, no structure "data" "123" "cache_item" ``` ### 2. Set Appropriate Expiration ```csharp // Session data - short expiration await cache.SetAsync("session:xyz", data, TimeSpan.FromMinutes(30)); // Reference data - longer expiration await cache.SetAsync("config:app", config, TimeSpan.FromHours(24)); // Computed data - based on freshness needs await cache.SetAsync("report:daily", report, TimeSpan.FromHours(1)); ``` ### 3. Handle Cache Misses Gracefully ```csharp var cached = await cache.GetAsync("user:123"); if (!cached.HasValue) { // Handle miss - load from source return await LoadFromDatabaseAsync(123); } // cached.Value is the User, cached.IsNull is true if explicitly cached as null ``` ### 4. Use Scoped Caches for Isolation ```csharp // Per-tenant isolation var tenantCache = new ScopedCacheClient(cache, $"tenant:{tenantId}"); // Per-feature isolation var featureCache = new ScopedCacheClient(cache, "feature:recommendations"); ``` ### 5. Consider Hybrid for High-Read Scenarios If you're doing many reads of the same data across instances, `HybridCacheClient` can dramatically reduce latency and Redis load. ## Next Steps * [Queues](./queues) - Message queuing for background processing * [Locks](./locks) - Distributed locking with cache-based implementation * [Redis Implementation](./implementations/redis) - Production Redis setup --- --- url: /guide/configuration.md --- # Configuration This guide covers configuration options for various Foundatio components. ## Cache Configuration ### InMemoryCacheClient ```csharp var cache = new InMemoryCacheClient(options => { // Maximum number of items (LRU eviction) options.MaxItems = 1000; // Clone values on get/set (default: true) options.CloneValues = true; // Expiration scan frequency options.ExpirationScanFrequency = TimeSpan.FromMinutes(1); // Logger options.LoggerFactory = loggerFactory; // Time provider for testing options.TimeProvider = timeProvider; }); ``` ### HybridCacheClient ```csharp var hybridCache = new HybridCacheClient( distributedCache: redisCacheClient, messageBus: redisMessageBus, options => { // Local cache max items options.LocalCacheMaxItems = 500; // Logger options.LoggerFactory = loggerFactory; } ); ``` ### ScopedCacheClient ```csharp var scopedCache = new ScopedCacheClient( cache: baseCacheClient, scope: "tenant:123" ); ``` ## Queue Configuration ### InMemoryQueue ```csharp var queue = new InMemoryQueue(options => { // Queue name/identifier options.Name = "work-items"; // Work item timeout options.WorkItemTimeout = TimeSpan.FromMinutes(5); // Retry settings options.Retries = 3; options.RetryDelay = TimeSpan.FromSeconds(30); // Logger options.LoggerFactory = loggerFactory; // Serializer options.Serializer = serializer; }); ``` ### RedisQueue ```csharp var queue = new RedisQueue(options => { // Redis connection options.ConnectionMultiplexer = redis; // Queue name options.Name = "work-items"; // Work item timeout options.WorkItemTimeout = TimeSpan.FromMinutes(5); // How often to check for dead items options.DeadLetterCheckInterval = TimeSpan.FromMinutes(1); // Maximum items per dead letter check options.DeadLetterMaxItems = 100; // Retry settings options.Retries = 3; options.RetryDelay = TimeSpan.FromSeconds(30); // Logger options.LoggerFactory = loggerFactory; }); ``` ## Messaging Configuration ### InMemoryMessageBus ```csharp var messageBus = new InMemoryMessageBus(options => { // Logger options.LoggerFactory = loggerFactory; // Serializer options.Serializer = serializer; }); ``` ### RedisMessageBus ```csharp var messageBus = new RedisMessageBus(options => { // Redis subscriber options.Subscriber = redis.GetSubscriber(); // Topic prefix options.Topic = "myapp"; // Logger options.LoggerFactory = loggerFactory; // Serializer options.Serializer = serializer; }); ``` ## Lock Configuration ### CacheLockProvider ```csharp var locker = new CacheLockProvider( cache: cacheClient, messageBus: messageBus, options => { // Default lock expiration options.DefaultTimeToLive = TimeSpan.FromMinutes(5); // Logger options.LoggerFactory = loggerFactory; } ); ``` ### ThrottlingLockProvider ```csharp var throttler = new ThrottlingLockProvider( cache: cacheClient, maxHits: 100, // Maximum operations period: TimeSpan.FromMinutes(1) // Per time period ); ``` ## Storage Configuration ### FolderFileStorage ```csharp var storage = new FolderFileStorage(options => { // Root folder path options.Folder = "/data/files"; // Logger options.LoggerFactory = loggerFactory; // Serializer options.Serializer = serializer; }); ``` ### InMemoryFileStorage ```csharp var storage = new InMemoryFileStorage(options => { // Maximum number of files options.MaxFiles = 1000; // Maximum total size options.MaxFileSize = 100 * 1024 * 1024; // 100MB // Logger options.LoggerFactory = loggerFactory; // Serializer options.Serializer = serializer; }); ``` ### AzureFileStorage ```csharp var storage = new AzureFileStorage(options => { // Connection string options.ConnectionString = "DefaultEndpointsProtocol=https;..."; // Container name options.ContainerName = "files"; // Logger options.LoggerFactory = loggerFactory; // Serializer options.Serializer = serializer; }); ``` ### S3FileStorage ```csharp var storage = new S3FileStorage(options => { // AWS region options.Region = RegionEndpoint.USEast1; // Bucket name options.Bucket = "my-files"; // Optional credentials (uses default chain if not specified) options.AccessKey = "..."; options.SecretKey = "..."; // Logger options.LoggerFactory = loggerFactory; }); ``` ## Resilience Configuration ### ResiliencePolicy ```csharp var policy = new ResiliencePolicy { // Maximum attempts MaxAttempts = 5, // Fixed delay between retries Delay = TimeSpan.FromSeconds(2), // Or exponential delay GetDelay = ResiliencePolicy.ExponentialDelay(TimeSpan.FromSeconds(1)), // Maximum delay cap MaxDelay = TimeSpan.FromMinutes(1), // Add jitter UseJitter = true, // Overall timeout Timeout = TimeSpan.FromMinutes(5), // Exceptions to not retry UnhandledExceptions = { typeof(OperationCanceledException) }, // Custom retry logic ShouldRetry = (attempt, ex) => ex is TransientException, // Logger Logger = logger }; ``` ### CircuitBreaker ```csharp var circuitBreaker = new CircuitBreakerBuilder() // Failure threshold (0.0 - 1.0) .WithFailureRatio(0.5) // Minimum calls before evaluating .WithMinimumCalls(10) // Duration to keep circuit open .WithBreakDuration(TimeSpan.FromMinutes(1)) // Success threshold for half-open recovery .WithSuccessThreshold(3) // Sampling duration for failure rate .WithSamplingDuration(TimeSpan.FromMinutes(5)) .Build(); ``` ## Job Configuration ### JobOptions ```csharp var options = new JobOptions { // Job name for logging Name = "CleanupJob", // Interval between runs Interval = TimeSpan.FromHours(1), // Maximum iterations (-1 for unlimited) IterationLimit = -1, // Initial run delay InitialDelay = TimeSpan.FromMinutes(5) }; await job.RunContinuousAsync(options, stoppingToken); ``` ### JobRunner ```csharp var runner = new JobRunner( job: myJob, instanceCount: 4, // Number of parallel instances interval: TimeSpan.FromSeconds(5) ); ``` ## Serialization Configuration ### Custom Serializer ```csharp // Using System.Text.Json var serializer = new SystemTextJsonSerializer(new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase, WriteIndented = false }); // Apply to services var cache = new InMemoryCacheClient(o => o.Serializer = serializer); var queue = new InMemoryQueue(o => o.Serializer = serializer); var storage = new InMemoryFileStorage(o => o.Serializer = serializer); ``` ## Logging Configuration ### Configure Logging ```csharp var loggerFactory = LoggerFactory.Create(builder => { builder .AddConsole() .SetMinimumLevel(LogLevel.Information) .AddFilter("Foundatio", LogLevel.Debug); }); // Apply to services var cache = new InMemoryCacheClient(o => o.LoggerFactory = loggerFactory); var queue = new InMemoryQueue(o => o.LoggerFactory = loggerFactory); ``` ## Environment Variables ### Common Environment Variables ```bash # Redis connection FOUNDATIO_REDIS_CONNECTION=localhost:6379 # Azure Storage FOUNDATIO_AZURE_STORAGE_CONNECTION=DefaultEndpointsProtocol=https;... # AWS AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... AWS_REGION=us-east-1 ``` ### Reading from Configuration ```csharp var builder = WebApplication.CreateBuilder(args); var redisConnection = builder.Configuration["Foundatio:Redis:Connection"]; var azureStorage = builder.Configuration["Foundatio:Azure:StorageConnection"]; builder.Services.AddSingleton(sp => { if (!string.IsNullOrEmpty(redisConnection)) { var redis = ConnectionMultiplexer.Connect(redisConnection); return new RedisCacheClient(o => o.ConnectionMultiplexer = redis); } return new InMemoryCacheClient(); }); ``` ## appsettings.json Example ```json { "Foundatio": { "Cache": { "Type": "Redis", "Connection": "localhost:6379", "MaxItems": 1000 }, "Queue": { "Type": "Redis", "WorkItemTimeout": "00:05:00", "Retries": 3 }, "Storage": { "Type": "Azure", "ConnectionString": "...", "ContainerName": "files" }, "Resilience": { "MaxAttempts": 5, "InitialDelay": "00:00:01", "UseJitter": true } } } ``` ### Reading Configuration ```csharp public class FoundatioOptions { public CacheOptions Cache { get; set; } public QueueOptions Queue { get; set; } public StorageOptions Storage { get; set; } public ResilienceOptions Resilience { get; set; } } public class CacheOptions { public string Type { get; set; } public string Connection { get; set; } public int MaxItems { get; set; } } // Registration builder.Services.Configure( builder.Configuration.GetSection("Foundatio")); ``` ## Best Practices ### 1. Use Options Pattern ```csharp // Define options class public class CacheOptions { public string Type { get; set; } = "InMemory"; public int MaxItems { get; set; } = 1000; public string RedisConnection { get; set; } } // Register and use builder.Services.Configure( builder.Configuration.GetSection("Cache")); builder.Services.AddSingleton(sp => { var options = sp.GetRequiredService>().Value; return options.Type == "Redis" ? new RedisCacheClient(...) : new InMemoryCacheClient(o => o.MaxItems = options.MaxItems); }); ``` ### 2. Validate Configuration ```csharp builder.Services.AddOptions() .Bind(builder.Configuration.GetSection("Cache")) .ValidateDataAnnotations() .ValidateOnStart(); ``` ### 3. Use Environment-Specific Settings ``` appsettings.json # Base settings appsettings.Development.json # In-memory implementations appsettings.Production.json # Redis/Azure implementations ``` ### 4. Keep Secrets Secure ```csharp // Use secrets manager var redis = builder.Configuration["Redis:ConnectionString"]; // Or environment variables var redis = Environment.GetEnvironmentVariable("REDIS_CONNECTION"); ``` ## Next Steps * [Dependency Injection](./dependency-injection) - Service registration patterns * [Getting Started](./getting-started) - Initial setup guide * [Caching](./caching) - Cache configuration details --- --- url: /guide/dependency-injection.md --- # Dependency Injection Foundatio is designed to work seamlessly with Microsoft.Extensions.DependencyInjection. All abstractions are interface-based and can be easily registered and resolved. ## Basic Registration ### Manual Registration ```csharp using Foundatio.Caching; using Foundatio.Messaging; using Foundatio.Lock; using Foundatio.Storage; using Foundatio.Queues; var builder = WebApplication.CreateBuilder(args); // Core services builder.Services.AddSingleton(); builder.Services.AddSingleton(); builder.Services.AddSingleton(); // Lock provider (depends on cache and message bus) builder.Services.AddSingleton(sp => new CacheLockProvider( sp.GetRequiredService(), sp.GetRequiredService() ) ); // Queues builder.Services.AddSingleton>(sp => new InMemoryQueue() ); ``` ### Using Extension Methods ```csharp using Foundatio; builder.Services.AddFoundatio(); // Adds default in-memory implementations ``` ## Service Lifetimes ### Recommended Lifetimes | Service | Lifetime | Reason | |---------|----------|--------| | `ICacheClient` | Singleton | Maintains internal state/connection | | `IMessageBus` | Singleton | Maintains subscriptions | | `ILockProvider` | Singleton | Stateless, thread-safe | | `IFileStorage` | Singleton | Stateless, thread-safe | | `IQueue` | Singleton | Maintains queue state | | Jobs | Scoped | Per-execution isolation | ### Example Registration ```csharp // Singletons for infrastructure builder.Services.AddSingleton(sp => new InMemoryCacheClient(o => o.MaxItems = 1000)); builder.Services.AddSingleton(); // Scoped for per-request isolation builder.Services.AddScoped(sp => new ScopedLockProvider( sp.GetRequiredService(), $"tenant:{GetCurrentTenantId(sp)}" ) ); ``` ## Environment-Based Configuration ### Development vs Production ```csharp if (builder.Environment.IsDevelopment()) { // In-memory for development builder.Services.AddSingleton(); builder.Services.AddSingleton(); builder.Services.AddSingleton(); } else { // Redis for production var redis = ConnectionMultiplexer.Connect( builder.Configuration.GetConnectionString("Redis") ); builder.Services.AddSingleton(redis); builder.Services.AddSingleton(sp => new RedisCacheClient(o => o.ConnectionMultiplexer = redis)); builder.Services.AddSingleton(sp => new RedisMessageBus(o => o.Subscriber = redis.GetSubscriber())); builder.Services.AddSingleton(sp => new AzureFileStorage(o => { o.ConnectionString = builder.Configuration["Azure:StorageConnectionString"]; o.ContainerName = "files"; })); } ``` ### Using Options Pattern ```csharp // appsettings.json { "Foundatio": { "Cache": { "Type": "Redis", "MaxItems": 1000 }, "Storage": { "Type": "Azure", "ContainerName": "files" } } } // Registration builder.Services.Configure( builder.Configuration.GetSection("Foundatio")); builder.Services.AddSingleton(sp => { var options = sp.GetRequiredService>().Value; return options.Cache.Type switch { "Redis" => new RedisCacheClient(...), "InMemory" => new InMemoryCacheClient(o => o.MaxItems = options.Cache.MaxItems), _ => throw new InvalidOperationException() }; }); ``` ## Named/Keyed Services ### Multiple Implementations ```csharp // Multiple caches builder.Services.AddKeyedSingleton("session", sp => new InMemoryCacheClient(o => o.MaxItems = 10000)); builder.Services.AddKeyedSingleton("data", sp => new RedisCacheClient(o => o.ConnectionMultiplexer = redis)); // Multiple queues builder.Services.AddKeyedSingleton>("high-priority", sp => new InMemoryQueue()); builder.Services.AddKeyedSingleton>("low-priority", sp => new InMemoryQueue()); ``` ### Injecting Keyed Services ```csharp public class OrderService { private readonly ICacheClient _sessionCache; private readonly ICacheClient _dataCache; public OrderService( [FromKeyedServices("session")] ICacheClient sessionCache, [FromKeyedServices("data")] ICacheClient dataCache) { _sessionCache = sessionCache; _dataCache = dataCache; } } ``` ## Factory Pattern ### Dynamic Resolution ```csharp public interface ICacheClientFactory { ICacheClient GetCache(string name); } public class CacheClientFactory : ICacheClientFactory { private readonly IServiceProvider _services; private readonly ConcurrentDictionary _caches = new(); public CacheClientFactory(IServiceProvider services) { _services = services; } public ICacheClient GetCache(string name) { return _caches.GetOrAdd(name, n => { var baseCache = _services.GetRequiredService(); return new ScopedCacheClient(baseCache, n); }); } } // Registration builder.Services.AddSingleton(); builder.Services.AddSingleton(); ``` ## Multi-Tenant Support ### Tenant-Scoped Services ```csharp public interface ITenantAccessor { string TenantId { get; } } // Scoped cache per tenant builder.Services.AddScoped(sp => { var baseCache = sp.GetRequiredService(); var tenant = sp.GetRequiredService(); return new ScopedCacheClient(baseCache, $"tenant:{tenant.TenantId}"); }); // Scoped storage per tenant builder.Services.AddScoped(sp => { var baseStorage = sp.GetRequiredService(); var tenant = sp.GetRequiredService(); return new ScopedFileStorage(baseStorage, tenant.TenantId); }); // Scoped locks per tenant builder.Services.AddScoped(sp => { var baseLock = sp.GetRequiredService(); var tenant = sp.GetRequiredService(); return new ScopedLockProvider(baseLock, tenant.TenantId); }); ``` ## Health Checks ### Register Health Checks ```csharp builder.Services.AddHealthChecks() .AddCheck("cache") .AddCheck("storage") .AddCheck("queue"); public class CacheHealthCheck : IHealthCheck { private readonly ICacheClient _cache; public CacheHealthCheck(ICacheClient cache) => _cache = cache; public async Task CheckHealthAsync( HealthCheckContext context, CancellationToken cancellationToken = default) { try { await _cache.SetAsync("health-check", DateTime.UtcNow); var result = await _cache.GetAsync("health-check"); return result.HasValue ? HealthCheckResult.Healthy() : HealthCheckResult.Unhealthy("Cache read failed"); } catch (Exception ex) { return HealthCheckResult.Unhealthy(ex.Message); } } } ``` ## Testing ### Test-Friendly Registration ```csharp // In test setup public class TestStartup { public void ConfigureServices(IServiceCollection services) { // Always use in-memory for tests services.AddSingleton(); services.AddSingleton(); services.AddSingleton(); services.AddSingleton(sp => new CacheLockProvider( sp.GetRequiredService(), sp.GetRequiredService() ) ); } } ``` ### Isolated Tests ```csharp public class OrderServiceTests { private readonly ServiceProvider _services; public OrderServiceTests() { var services = new ServiceCollection(); // Fresh instances for each test class services.AddSingleton(); services.AddSingleton(); _services = services.BuildServiceProvider(); } [Fact] public async Task CreateOrder_CachesOrder() { var cache = _services.GetRequiredService(); var service = new OrderService(cache); var order = await service.CreateOrderAsync(new CreateOrderRequest()); var cached = await cache.GetAsync($"order:{order.Id}"); Assert.True(cached.HasValue); } } ``` ## Best Practices ### 1. Use Interfaces for Dependencies ```csharp // ✅ Good: Interface dependency public class OrderService { private readonly ICacheClient _cache; public OrderService(ICacheClient cache) { _cache = cache; } } // ❌ Bad: Concrete dependency public class OrderService { private readonly RedisCacheClient _cache; // Harder to test } ``` ### 2. Avoid Service Locator Pattern ```csharp // ✅ Good: Constructor injection public class MyService { private readonly ICacheClient _cache; public MyService(ICacheClient cache) { _cache = cache; } } // ❌ Bad: Service locator public class MyService { private readonly IServiceProvider _services; public void DoWork() { var cache = _services.GetService(); } } ``` ### 3. Register as Singletons When Appropriate ```csharp // Stateless services that maintain connections builder.Services.AddSingleton(...); builder.Services.AddSingleton(...); // Not scoped unless you need tenant isolation ``` ### 4. Validate Configuration at Startup ```csharp builder.Services.AddSingleton(sp => { var connectionString = builder.Configuration["Redis:ConnectionString"]; if (string.IsNullOrEmpty(connectionString)) throw new InvalidOperationException("Redis connection string not configured"); var redis = ConnectionMultiplexer.Connect(connectionString); return new RedisCacheClient(o => o.ConnectionMultiplexer = redis); }); ``` ## Next Steps * [Configuration](./configuration) - Configuration options for Foundatio services * [Caching](./caching) - Deep dive into caching * [Getting Started](./getting-started) - Initial setup guide --- --- url: /guide/storage.md --- # File Storage File storage provides abstracted file operations with multiple backend implementations. Foundatio's `IFileStorage` interface allows you to work with files consistently across local disk, cloud storage, and more. ## The IFileStorage Interface ```csharp public interface IFileStorage : IHaveSerializer, IDisposable { Task GetFileStreamAsync(string path, StreamMode streamMode, CancellationToken cancellationToken = default); Task GetFileInfoAsync(string path); Task ExistsAsync(string path); Task SaveFileAsync(string path, Stream stream, CancellationToken cancellationToken = default); Task RenameFileAsync(string path, string newPath, CancellationToken cancellationToken = default); Task CopyFileAsync(string path, string targetPath, CancellationToken cancellationToken = default); Task DeleteFileAsync(string path, CancellationToken cancellationToken = default); Task DeleteFilesAsync(string searchPattern = null, CancellationToken cancellation = default); Task GetPagedFileListAsync(int pageSize = 100, string searchPattern = null, CancellationToken cancellationToken = default); } ``` ## Implementations ### InMemoryFileStorage An in-memory storage for development and testing: ```csharp using Foundatio.Storage; var storage = new InMemoryFileStorage(); // Save a file await storage.SaveFileAsync("documents/report.pdf", pdfStream); // Read a file var stream = await storage.GetFileStreamAsync("documents/report.pdf", StreamMode.Read); ``` ### FolderFileStorage File storage backed by the local file system: ```csharp using Foundatio.Storage; var storage = new FolderFileStorage(o => o.Folder = "/data/files"); // Files are stored in /data/files/documents/report.pdf await storage.SaveFileAsync("documents/report.pdf", pdfStream); ``` ### ScopedFileStorage Prefix all paths with a scope: ```csharp using Foundatio.Storage; var baseStorage = new FolderFileStorage(o => o.Folder = "/data"); var tenantStorage = new ScopedFileStorage(baseStorage, "tenant-abc"); // Path becomes: tenant-abc/documents/report.pdf await tenantStorage.SaveFileAsync("documents/report.pdf", pdfStream); ``` ### AzureFileStorage Azure Blob Storage (separate package): ```csharp // dotnet add package Foundatio.AzureStorage using Foundatio.AzureStorage.Storage; var storage = new AzureFileStorage(o => { o.ConnectionString = "DefaultEndpointsProtocol=https;..."; o.ContainerName = "files"; }); ``` ### S3FileStorage AWS S3 Storage (separate package): ```csharp // dotnet add package Foundatio.AWS using Foundatio.AWS.Storage; var storage = new S3FileStorage(o => { o.Region = RegionEndpoint.USEast1; o.Bucket = "my-files"; }); ``` ### RedisFileStorage Redis-backed storage (separate package): ```csharp // dotnet add package Foundatio.Redis using Foundatio.Redis.Storage; var storage = new RedisFileStorage(o => { o.ConnectionMultiplexer = redis; }); ``` ### MinioFileStorage Minio object storage (separate package): ```csharp // dotnet add package Foundatio.Minio using Foundatio.Minio.Storage; var storage = new MinioFileStorage(o => { o.Endpoint = "localhost:9000"; o.AccessKey = "minioadmin"; o.SecretKey = "minioadmin"; o.Bucket = "files"; }); ``` ### SshNetFileStorage SFTP-backed storage (separate package): ```csharp // dotnet add package Foundatio.Storage.SshNet using Foundatio.Storage.SshNet; var storage = new SshNetFileStorage(o => { o.Host = "sftp.example.com"; o.Username = "user"; o.Password = "password"; o.WorkingDirectory = "/uploads"; }); ``` ## Basic Operations ### Saving Files ```csharp var storage = new InMemoryFileStorage(); // Save from stream using var stream = File.OpenRead("local-file.pdf"); await storage.SaveFileAsync("remote/file.pdf", stream); // Save string content (extension method) await storage.SaveFileAsync("config.json", """{"key": "value"}"""); // Save with object serialization (extension method) await storage.SaveObjectAsync("data/user.json", new User { Name = "John" }); ``` ### Reading Files ```csharp // Get file stream for reading using var stream = await storage.GetFileStreamAsync("file.pdf", StreamMode.Read); // Read as string (extension method) string content = await storage.GetFileContentsAsync("config.json"); // Read and deserialize (extension method) var user = await storage.GetObjectAsync("data/user.json"); // Get raw bytes (extension method) byte[] bytes = await storage.GetFileBytesAsync("image.png"); ``` ### File Information ```csharp // Check if file exists bool exists = await storage.ExistsAsync("file.pdf"); // Get file info var fileSpec = await storage.GetFileInfoAsync("file.pdf"); if (fileSpec != null) { Console.WriteLine($"Path: {fileSpec.Path}"); Console.WriteLine($"Size: {fileSpec.Size} bytes"); Console.WriteLine($"Modified: {fileSpec.Modified}"); Console.WriteLine($"Created: {fileSpec.Created}"); } ``` ### Modifying Files ```csharp // Rename/move file await storage.RenameFileAsync("old/path.pdf", "new/path.pdf"); // Copy file await storage.CopyFileAsync("source.pdf", "backup/source.pdf"); // Delete file await storage.DeleteFileAsync("file.pdf"); // Delete multiple files by pattern int deleted = await storage.DeleteFilesAsync("temp/*"); ``` ### Listing Files ```csharp // List all files var files = await storage.GetFileListAsync(); foreach (var file in files) { Console.WriteLine($"{file.Path} - {file.Size} bytes"); } // List with pattern var pdfFiles = await storage.GetFileListAsync("documents/*.pdf"); // Paged listing for large directories var result = await storage.GetPagedFileListAsync(pageSize: 100, "logs/*"); do { foreach (var file in result.Files) { Console.WriteLine(file.Path); } } while (await result.NextPageAsync()); ``` ## Stream Modes Control how file streams are opened: ```csharp // Read mode - for reading existing files using var readStream = await storage.GetFileStreamAsync("file.pdf", StreamMode.Read); // Write mode - for creating/overwriting files using var writeStream = await storage.GetFileStreamAsync("file.pdf", StreamMode.Write); await someData.CopyToAsync(writeStream); ``` ## Common Patterns ### File Upload/Download ```csharp public class FileService { private readonly IFileStorage _storage; public async Task UploadAsync(IFormFile file, string folder) { var path = $"{folder}/{Guid.NewGuid()}{Path.GetExtension(file.FileName)}"; using var stream = file.OpenReadStream(); await _storage.SaveFileAsync(path, stream); return path; } public async Task DownloadAsync(string path) { if (!await _storage.ExistsAsync(path)) throw new FileNotFoundException(path); return await _storage.GetFileStreamAsync(path, StreamMode.Read); } } ``` ### Organized File Structure ```csharp public class DocumentStorage { private readonly IFileStorage _storage; public string GetPath(int tenantId, int documentId, string fileName) { // Organized path: tenants/{id}/documents/{year}/{month}/{id}/{filename} var now = DateTime.UtcNow; return $"tenants/{tenantId}/documents/{now:yyyy}/{now:MM}/{documentId}/{fileName}"; } public async Task SaveDocumentAsync(int tenantId, int documentId, string fileName, Stream content) { var path = GetPath(tenantId, documentId, fileName); await _storage.SaveFileAsync(path, content); } } ``` ### File Versioning ```csharp public class VersionedFileStorage { private readonly IFileStorage _storage; public async Task SaveVersionAsync(string basePath, Stream content) { var version = DateTime.UtcNow.ToString("yyyyMMddHHmmss"); var versionPath = $"{basePath}.v{version}"; // Save new version await _storage.SaveFileAsync(versionPath, content); // Update "current" pointer await _storage.CopyFileAsync(versionPath, basePath); } public async Task> GetVersionsAsync(string basePath) { return await _storage.GetFileListAsync($"{basePath}.v*"); } } ``` ### Temporary File Cleanup ```csharp public class TempFileCleanup { private readonly IFileStorage _storage; public async Task CleanupOldFilesAsync(TimeSpan maxAge) { var files = await _storage.GetFileListAsync("temp/*"); var cutoff = DateTime.UtcNow - maxAge; foreach (var file in files) { if (file.Modified < cutoff) { await _storage.DeleteFileAsync(file.Path); } } } } ``` ### Multi-Tenant Storage ```csharp public class TenantStorageFactory { private readonly IFileStorage _baseStorage; public IFileStorage GetStorageForTenant(string tenantId) { return new ScopedFileStorage(_baseStorage, $"tenants/{tenantId}"); } } // Usage var tenantStorage = _storageFactory.GetStorageForTenant("tenant-123"); await tenantStorage.SaveFileAsync("documents/report.pdf", stream); // Actual path: tenants/tenant-123/documents/report.pdf ``` ## Extension Methods Foundatio provides helpful extension methods: ```csharp // String content await storage.SaveFileAsync("text.txt", "Hello World"); string text = await storage.GetFileContentsAsync("text.txt"); // Bytes await storage.SaveFileAsync("data.bin", byteArray); byte[] bytes = await storage.GetFileBytesAsync("data.bin"); // Objects (serialized) await storage.SaveObjectAsync("user.json", user); var user = await storage.GetObjectAsync("user.json"); // Non-paged file listing var allFiles = await storage.GetFileListAsync(); var filtered = await storage.GetFileListAsync("*.pdf"); ``` ## Dependency Injection ### Basic Registration ```csharp // In-memory (development) services.AddSingleton(); // Folder (local development with persistence) services.AddSingleton(sp => new FolderFileStorage(o => o.Folder = "./storage") ); // Azure Blob (production) services.AddSingleton(sp => new AzureFileStorage(o => { o.ConnectionString = configuration["Azure:StorageConnectionString"]; o.ContainerName = "files"; }) ); ``` ### With Scoping ```csharp services.AddSingleton(sp => new FolderFileStorage(o => o.Folder = "/data") ); // Scoped storage per tenant services.AddScoped((sp, tenantId) => { var baseStorage = sp.GetRequiredService(); return new ScopedFileStorage(baseStorage, $"tenant:{tenantId}"); }); ``` ## Best Practices ### 1. Use Meaningful Paths ```csharp // ✅ Good: Organized, meaningful paths "documents/invoices/2024/01/invoice-12345.pdf" "users/user-123/avatars/profile.jpg" "temp/uploads/session-abc/file.tmp" // ❌ Bad: Flat, unclear paths "file1.pdf" "12345.pdf" "abc123" ``` ### 2. Include Extension in Path ```csharp // ✅ Good: Extension present await storage.SaveFileAsync("report.pdf", stream); // ❌ Bad: No extension await storage.SaveFileAsync("report", stream); ``` ### 3. Use Scoped Storage for Isolation ```csharp // Each tenant has isolated storage var tenantStorage = new ScopedFileStorage(baseStorage, tenantId); ``` ### 4. Handle Missing Files ```csharp if (!await storage.ExistsAsync(path)) { throw new FileNotFoundException($"File not found: {path}"); } var stream = await storage.GetFileStreamAsync(path, StreamMode.Read); ``` ### 5. Dispose Streams Properly ```csharp // ✅ Good: Using statement using var stream = await storage.GetFileStreamAsync(path, StreamMode.Read); await stream.CopyToAsync(destination); // ❌ Bad: Not disposing var stream = await storage.GetFileStreamAsync(path, StreamMode.Read); await stream.CopyToAsync(destination); // stream never disposed! ``` ### 6. Use Appropriate Storage for Use Case | Use Case | Recommended Storage | |----------|---------------------| | Development/Testing | InMemoryFileStorage | | Local persistence | FolderFileStorage | | Cloud applications | AzureFileStorage, S3FileStorage | | Multi-cloud | MinioFileStorage | | Legacy systems | SshNetFileStorage | ## FileSpec Properties ```csharp public class FileSpec { public string Path { get; set; } public long Size { get; set; } public DateTime Created { get; set; } public DateTime Modified { get; set; } } ``` ## Next Steps * [Caching](./caching) - Cache file metadata * [Jobs](./jobs) - Background file processing * [Azure Implementation](./implementations/azure) - Production Azure setup * [AWS Implementation](./implementations/aws) - Production AWS setup --- --- url: /README.md --- # Foundatio Documentation This is the documentation site for [Foundatio](https://github.com/FoundatioFx/Foundatio), built with [VitePress](https://vitepress.dev). ## Prerequisites * [Node.js](https://nodejs.org/) 18.x or higher * npm (comes with Node.js) ## Getting Started ### Install Dependencies ```bash npm install ``` ### Development Start the development server with hot-reload: ```bash npm run docs:dev ``` The site will be available at `http://localhost:5173`. ### Build Build the static site for production: ```bash npm run docs:build ``` The built files will be in `.vitepress/dist`. ### Preview Preview the production build locally: ```bash npm run docs:preview ``` ## Project Structure ``` docs/ ├── .vitepress/ │ ├── config.ts # VitePress configuration │ └── dist/ # Built output (generated) ├── guide/ │ ├── what-is-foundatio.md │ ├── getting-started.md │ ├── why-foundatio.md │ ├── caching.md │ ├── queues.md │ ├── locks.md │ ├── messaging.md │ ├── storage.md │ ├── jobs.md │ ├── resilience.md │ ├── dependency-injection.md │ ├── configuration.md │ └── implementations/ │ ├── in-memory.md │ ├── redis.md │ ├── azure.md │ └── aws.md ├── index.md # Home page ├── package.json └── README.md # This file ``` ## Writing Documentation ### Adding New Pages 1. Create a new `.md` file in the appropriate directory 2. Add frontmatter with title: ```yaml --- title: Your Page Title --- ``` 3. Add the page to the sidebar in `.vitepress/config.ts` ### Code Blocks Use triple backticks with language identifier: ````markdown ```csharp var cache = new InMemoryCacheClient(); await cache.SetAsync("key", "value"); ``` ```` ### Admonitions VitePress supports custom containers: ```markdown ::: info Informational message ::: ::: tip Helpful tip ::: ::: warning Warning message ::: ::: danger Critical warning ::: ``` ### Internal Links Use relative paths for internal links: ```markdown [Getting Started](./getting-started) [Caching Guide](./caching.md) ``` ## Plugins This documentation site uses: * **vitepress-plugin-llms**: Generates LLM-friendly documentation at `/llms.txt` * **vitepress-plugin-mermaid**: Enables Mermaid diagrams in markdown ### Mermaid Diagrams ````markdown ```mermaid graph LR A[Client] --> B[Cache] B --> C[Redis] ```` ``` ## Deployment The documentation can be deployed to any static hosting service: - GitHub Pages - Netlify - Vercel - Azure Static Web Apps - AWS S3 + CloudFront ### GitHub Pages 1. Build the site: `npm run docs:build` 2. Deploy `.vitepress/dist` to `gh-pages` branch ### Netlify / Vercel Connect your repository and configure: - Build command: `npm run docs:build` - Output directory: `.vitepress/dist` ## Contributing 1. Fork the repository 2. Create a feature branch 3. Make your changes 4. Run the dev server to preview 5. Submit a pull request ## License This documentation is part of the Foundatio project and is licensed under the same terms. ``` --- --- url: /guide/getting-started.md --- # Getting Started This guide will walk you through installing Foundatio and using your first abstractions. ## Installation Foundatio is available on [NuGet](https://www.nuget.org/packages?q=Foundatio). Install the core package: ```bash dotnet add package Foundatio ``` For specific implementations, install the corresponding packages: ```bash # Redis implementations dotnet add package Foundatio.Redis # Azure Storage (Queues, Blobs) dotnet add package Foundatio.AzureStorage # Azure Service Bus (Queues, Messaging) dotnet add package Foundatio.AzureServiceBus # AWS (SQS, S3) dotnet add package Foundatio.AWS # RabbitMQ (Messaging) dotnet add package Foundatio.RabbitMQ # Kafka (Messaging) dotnet add package Foundatio.Kafka ``` ## Basic Setup ### 1. Register Services Configure Foundatio services in your application's dependency injection container: ```csharp using Foundatio.Caching; using Foundatio.Messaging; using Foundatio.Lock; using Foundatio.Storage; using Foundatio.Queues; var builder = WebApplication.CreateBuilder(args); // Register core services builder.Services.AddSingleton(); builder.Services.AddSingleton(); builder.Services.AddSingleton(); // Register lock provider (depends on cache and message bus) builder.Services.AddSingleton(sp => new CacheLockProvider( sp.GetRequiredService(), sp.GetRequiredService() ) ); // Register queues builder.Services.AddSingleton>(sp => new InMemoryQueue() ); var app = builder.Build(); ``` ### 2. Use the Services Inject and use the services in your application: ```csharp public class OrderService { private readonly ICacheClient _cache; private readonly IQueue _queue; private readonly ILockProvider _locker; private readonly IMessageBus _messageBus; public OrderService( ICacheClient cache, IQueue queue, ILockProvider locker, IMessageBus messageBus) { _cache = cache; _queue = queue; _locker = locker; _messageBus = messageBus; } public async Task CreateOrderAsync(CreateOrderRequest request) { // Use distributed lock to prevent duplicate orders await using var @lock = await _locker.AcquireAsync($"order:{request.CustomerId}"); if (@lock == null) throw new InvalidOperationException("Could not acquire lock"); // Create order var order = new Order { Id = Guid.NewGuid(), CustomerId = request.CustomerId }; // Cache the order await _cache.SetAsync($"order:{order.Id}", order, TimeSpan.FromHours(1)); // Queue for background processing await _queue.EnqueueAsync(new OrderWorkItem { OrderId = order.Id }); // Publish event for other services await _messageBus.PublishAsync(new OrderCreatedEvent { OrderId = order.Id }); return order; } } ``` ## Switching to Production Implementations When moving to production, swap in-memory implementations for distributed ones: ```csharp using Foundatio.Redis.Cache; using Foundatio.Redis.Messaging; using Foundatio.Redis.Queues; using StackExchange.Redis; var builder = WebApplication.CreateBuilder(args); // Configure Redis connection var redis = await ConnectionMultiplexer.ConnectAsync("localhost:6379"); builder.Services.AddSingleton(redis); // Use Redis implementations builder.Services.AddSingleton(sp => new RedisCacheClient(o => o.ConnectionMultiplexer = redis) ); builder.Services.AddSingleton(sp => new RedisMessageBus(o => o.Subscriber = redis.GetSubscriber()) ); builder.Services.AddSingleton>(sp => new RedisQueue(o => o.ConnectionMultiplexer = redis) ); ``` Your application code remains unchanged - only the DI registration changes! ## Working with Extension Methods Foundatio provides convenient extension methods through `FoundatioServicesExtensions`: ```csharp using Foundatio; var builder = WebApplication.CreateBuilder(args); // Add Foundatio with default in-memory implementations builder.Services.AddFoundatio(); // Or configure with options builder.Services.AddFoundatio(options => { options.UseInMemoryCache(); options.UseInMemoryMessageBus(); options.UseInMemoryQueues(); options.UseInMemoryStorage(); }); ``` ## Sample Application Here's a complete example showing all major abstractions working together: ```csharp using Foundatio.Caching; using Foundatio.Lock; using Foundatio.Messaging; using Foundatio.Queues; using Foundatio.Storage; // Setup services var cache = new InMemoryCacheClient(); var messageBus = new InMemoryMessageBus(); var storage = new InMemoryFileStorage(); var locker = new CacheLockProvider(cache, messageBus); var queue = new InMemoryQueue(); // Subscribe to messages await messageBus.SubscribeAsync(msg => { Console.WriteLine($"Work completed: {msg.ItemId}"); }); // Store a file await storage.SaveFileAsync("config.json", """{"setting": "value"}"""); // Queue work await queue.EnqueueAsync(new WorkItem { Id = "item-1" }); // Process queue with locking while (true) { var entry = await queue.DequeueAsync(TimeSpan.FromSeconds(5)); if (entry == null) break; // Acquire lock for this item await using var @lock = await locker.AcquireAsync($"work:{entry.Value.Id}"); if (@lock != null) { // Cache progress await cache.SetAsync($"progress:{entry.Value.Id}", "processing"); // Do work... // Complete entry await entry.CompleteAsync(); // Publish completion event await messageBus.PublishAsync(new WorkCompleted { ItemId = entry.Value.Id }); } else { // Couldn't get lock, abandon for retry await entry.AbandonAsync(); } } public record WorkItem { public string Id { get; init; } } public record WorkCompleted { public string ItemId { get; init; } } ``` ## Next Steps Now that you have the basics working, explore more advanced features: * [Caching](./caching) - Deep dive into caching patterns * [Queues](./queues) - Queue processing and behaviors * [Locks](./locks) - Distributed locking strategies * [Messaging](./messaging) - Pub/sub patterns * [Storage](./storage) - File storage operations * [Jobs](./jobs) - Background job processing * [Resilience](./resilience) - Retry policies and circuit breakers ## LLM-Friendly Documentation For AI assistants and Large Language Models, we provide optimized documentation formats: * [📜 LLMs Index](/llms.txt) - Quick reference with links to all sections * [📖 Complete Documentation](/llms-full.txt) - All docs in one LLM-friendly file These files follow the [llmstxt.org](https://llmstxt.org/) standard and contain the same information as this documentation in a format optimized for AI consumption. --- --- url: /guide/implementations/in-memory.md --- # In-Memory Implementations Foundatio provides in-memory implementations for all core abstractions. These are perfect for development, testing, and single-process applications. ## Overview | Implementation | Interface | Package | |----------------|-----------|---------| | `InMemoryCacheClient` | `ICacheClient` | Foundatio | | `InMemoryQueue` | `IQueue` | Foundatio | | `InMemoryMessageBus` | `IMessageBus` | Foundatio | | `InMemoryFileStorage` | `IFileStorage` | Foundatio | | `CacheLockProvider` | `ILockProvider` | Foundatio | ## Installation In-memory implementations are included in the core Foundatio package: ```bash dotnet add package Foundatio ``` ## InMemoryCacheClient A high-performance in-memory cache with optional LRU eviction. ### Basic Usage ```csharp using Foundatio.Caching; var cache = new InMemoryCacheClient(); // Store and retrieve values await cache.SetAsync("key", "value"); var value = await cache.GetAsync("key"); // With expiration await cache.SetAsync("temp", "data", TimeSpan.FromMinutes(5)); ``` ### Configuration Options ```csharp var cache = new InMemoryCacheClient(options => { // Maximum items (enables LRU eviction) options.MaxItems = 1000; // Clone values on get/set (thread safety) options.CloneValues = true; // How often to scan for expired items options.ExpirationScanFrequency = TimeSpan.FromMinutes(1); // Logger factory options.LoggerFactory = loggerFactory; // Time provider (useful for testing) options.TimeProvider = TimeProvider.System; }); ``` ### Features * **LRU Eviction**: Automatically removes least recently used items when `MaxItems` is reached * **Expiration**: Items can have absolute or sliding expiration * **Value Cloning**: Optionally clone values to prevent reference sharing issues * **Thread-Safe**: All operations are thread-safe ### DI Registration ```csharp // Simple registration services.AddSingleton(); // With configuration services.AddSingleton(sp => new InMemoryCacheClient(options => { options.MaxItems = 1000; options.LoggerFactory = sp.GetRequiredService(); })); ``` ## InMemoryQueue A thread-safe in-memory queue with retry support and dead letter handling. ### Basic Usage ```csharp using Foundatio.Queues; var queue = new InMemoryQueue(); // Enqueue items await queue.EnqueueAsync(new WorkItem { Id = 1, Data = "Hello" }); // Dequeue and process var entry = await queue.DequeueAsync(); if (entry != null) { // Process the item Console.WriteLine(entry.Value.Data); // Mark as complete await entry.CompleteAsync(); } ``` ### Configuration Options ```csharp var queue = new InMemoryQueue(options => { // Queue identifier options.Name = "work-items"; // Work item timeout (for retry) options.WorkItemTimeout = TimeSpan.FromMinutes(5); // Retry settings options.Retries = 3; options.RetryDelay = TimeSpan.FromSeconds(30); // Processing behaviors options.Behaviors.Add(new EnqueueAbandonedQueueEntryBehavior()); // Logger options.LoggerFactory = loggerFactory; }); ``` ### Processing Patterns ```csharp // Continuous processing with handler await queue.StartWorkingAsync(async (entry, token) => { await ProcessWorkItemAsync(entry.Value); }); // Process until empty while (await queue.GetQueueStatsAsync() is { Queued: > 0 }) { var entry = await queue.DequeueAsync(); await entry.CompleteAsync(); } ``` ### DI Registration ```csharp services.AddSingleton>(sp => new InMemoryQueue(options => { options.Name = "work-items"; options.WorkItemTimeout = TimeSpan.FromMinutes(5); options.LoggerFactory = sp.GetRequiredService(); })); ``` ## InMemoryMessageBus A simple in-memory pub/sub message bus for single-process communication. ### Basic Usage ```csharp using Foundatio.Messaging; var messageBus = new InMemoryMessageBus(); // Subscribe to messages await messageBus.SubscribeAsync(message => { Console.WriteLine($"User created: {message.UserId}"); }); // Publish messages await messageBus.PublishAsync(new UserCreatedEvent { UserId = "123" }); ``` ### Configuration Options ```csharp var messageBus = new InMemoryMessageBus(options => { options.LoggerFactory = loggerFactory; options.Serializer = serializer; }); ``` ### Subscription Management ```csharp // Subscribe with options await messageBus.SubscribeAsync( handler: async (message, token) => { await ProcessOrderAsync(message); }, cancellationToken: stoppingToken); // Type hierarchy subscription await messageBus.SubscribeAsync(message => { // Receives all events that inherit from BaseEvent }); ``` ### DI Registration ```csharp services.AddSingleton(); services.AddSingleton(sp => sp.GetRequiredService()); services.AddSingleton(sp => sp.GetRequiredService()); ``` ## InMemoryFileStorage An in-memory file storage implementation with optional size limits. ### Basic Usage ```csharp using Foundatio.Storage; var storage = new InMemoryFileStorage(); // Save files await storage.SaveFileAsync("documents/file.txt", "Hello, World!"); // Read files var content = await storage.GetFileContentsAsync("documents/file.txt"); // List files var files = await storage.GetFileListAsync("documents/"); ``` ### Configuration Options ```csharp var storage = new InMemoryFileStorage(options => { // Maximum number of files options.MaxFiles = 1000; // Maximum file size options.MaxFileSize = 100 * 1024 * 1024; // 100MB // Logger options.LoggerFactory = loggerFactory; // Serializer for metadata options.Serializer = serializer; }); ``` ### File Operations ```csharp // Save from stream using var stream = File.OpenRead("local-file.txt"); await storage.SaveFileAsync("remote/file.txt", stream); // Get file info var spec = await storage.GetFileInfoAsync("remote/file.txt"); Console.WriteLine($"Size: {spec?.Size}, Modified: {spec?.Modified}"); // Check existence if (await storage.ExistsAsync("remote/file.txt")) { // File exists } // Delete files await storage.DeleteFileAsync("remote/file.txt"); await storage.DeleteFilesAsync("temp/"); // Delete by pattern ``` ### DI Registration ```csharp services.AddSingleton(sp => new InMemoryFileStorage(options => { options.MaxFiles = 1000; options.LoggerFactory = sp.GetRequiredService(); })); ``` ## CacheLockProvider (In-Memory Locks) Use `CacheLockProvider` with `InMemoryCacheClient` for in-memory distributed locks. ### Basic Usage ```csharp using Foundatio.Lock; var cache = new InMemoryCacheClient(); var messageBus = new InMemoryMessageBus(); var locker = new CacheLockProvider(cache, messageBus); // Acquire a lock await using var lockHandle = await locker.AcquireAsync("resource-key"); if (lockHandle != null) { // Do exclusive work } ``` ### Configuration Options ```csharp var locker = new CacheLockProvider(cache, messageBus, options => { // Default lock duration options.DefaultTimeToLive = TimeSpan.FromMinutes(5); // Logger options.LoggerFactory = loggerFactory; }); ``` ### DI Registration ```csharp services.AddSingleton(sp => new CacheLockProvider( sp.GetRequiredService(), sp.GetRequiredService())); ``` ## Complete In-Memory Setup ### All Services ```csharp public static IServiceCollection AddFoundatioInMemory( this IServiceCollection services) { // Cache services.AddSingleton(); // Message Bus services.AddSingleton(); services.AddSingleton(sp => sp.GetRequiredService()); services.AddSingleton(sp => sp.GetRequiredService()); // Lock Provider services.AddSingleton(sp => new CacheLockProvider( sp.GetRequiredService(), sp.GetRequiredService())); // File Storage services.AddSingleton(); return services; } // With queues public static IServiceCollection AddFoundatioQueue( this IServiceCollection services, string name) where T : class { services.AddSingleton>(sp => new InMemoryQueue(options => { options.Name = name; options.LoggerFactory = sp.GetRequiredService(); })); return services; } ``` ### Usage ```csharp var builder = WebApplication.CreateBuilder(args); builder.Services.AddFoundatioInMemory(); builder.Services.AddFoundatioQueue("work-items"); builder.Services.AddFoundatioQueue("emails"); var app = builder.Build(); ``` ## When to Use In-Memory ### ✅ Good Use Cases * **Local Development**: No external dependencies needed * **Unit Testing**: Fast, isolated tests * **Single-Process Apps**: Simple deployments * **Prototyping**: Quick iterations * **Small Workloads**: Low-traffic applications ### ⚠️ Limitations * **No Persistence**: Data lost on restart * **Single Process**: No cross-process communication * **Memory Bound**: Limited by available RAM * **No Clustering**: Not suitable for distributed systems ### Switching to Production ```csharp // Easy to swap implementations if (builder.Environment.IsDevelopment()) { services.AddSingleton(); } else { services.AddSingleton(sp => new RedisCacheClient(options => { options.ConnectionMultiplexer = ConnectionMultiplexer.Connect("redis:6379"); })); } ``` ## Testing with In-Memory ```csharp public class CacheTests { [Fact] public async Task ShouldCacheAndRetrieveValue() { // Arrange var cache = new InMemoryCacheClient(); // Act await cache.SetAsync("key", "value"); var result = await cache.GetAsync("key"); // Assert Assert.Equal("value", result); } } ``` ## Next Steps * [Redis Implementation](./redis) - Production-ready distributed caching * [Azure Implementation](./azure) - Cloud-native Azure services * [AWS Implementation](./aws) - Amazon Web Services integration --- --- url: /guide/jobs.md --- # Jobs Jobs allow you to run long-running processes without worrying about them being terminated prematurely. Foundatio provides several patterns for defining and running jobs. ## The IJob Interface ```csharp public interface IJob { Task RunAsync(CancellationToken cancellationToken = default); } ``` ## Job Types Foundatio provides three main patterns for defining jobs: 1. **Standard Jobs** - Simple jobs that run independently 2. **Queue Processor Jobs** - Jobs that process items from a queue 3. **Work Item Jobs** - Jobs that handle work items from a shared pool ## Standard Jobs ### Basic Job Create a job by implementing `IJob` or deriving from `JobBase`: ```csharp using Foundatio.Jobs; public class CleanupJob : JobBase { private readonly ILogger _logger; public CleanupJob(ILogger logger) { _logger = logger; } protected override async Task RunInternalAsync(JobContext context) { _logger.LogInformation("Starting cleanup..."); // Do cleanup work var deletedCount = await CleanupOldRecordsAsync(context.CancellationToken); _logger.LogInformation("Cleaned up {Count} records", deletedCount); return JobResult.Success; } } ``` ### Running Jobs ```csharp var job = new CleanupJob(logger); // Run once await job.RunAsync(); // Run continuously with interval await job.RunContinuousAsync( interval: TimeSpan.FromMinutes(5), cancellationToken: stoppingToken ); // Run with iteration limit await job.RunContinuousAsync( iterationLimit: 100, cancellationToken: stoppingToken ); ``` ### Job Results ```csharp protected override Task RunInternalAsync(JobContext context) { try { // Success return Task.FromResult(JobResult.Success); // Success with message return Task.FromResult(JobResult.SuccessWithMessage("Processed 100 items")); // Failed return Task.FromResult(JobResult.Failed); // Failed with message return Task.FromResult(JobResult.FailedWithMessage("Database connection failed")); // Cancelled return Task.FromResult(JobResult.Cancelled); } catch (Exception ex) { // From exception return Task.FromResult(JobResult.FromException(ex)); } } ``` ## Queue Processor Jobs Process items from a queue automatically: ```csharp using Foundatio.Jobs; using Foundatio.Queues; public class OrderProcessorJob : QueueJobBase { private readonly IOrderService _orderService; private readonly ILogger _logger; public OrderProcessorJob( IQueue queue, IOrderService orderService, ILogger logger) : base(queue) { _orderService = orderService; _logger = logger; } protected override async Task ProcessQueueEntryAsync( QueueEntryContext context) { var workItem = context.QueueEntry.Value; _logger.LogInformation("Processing order {OrderId}", workItem.OrderId); try { await _orderService.ProcessAsync(workItem.OrderId, context.CancellationToken); return JobResult.Success; } catch (Exception ex) { _logger.LogError(ex, "Failed to process order {OrderId}", workItem.OrderId); return JobResult.FromException(ex); } } } public record OrderWorkItem { public int OrderId { get; init; } } ``` ### Running Queue Jobs ```csharp // Setup var queue = new InMemoryQueue(); var job = new OrderProcessorJob(queue, orderService, logger); // Enqueue work await queue.EnqueueAsync(new OrderWorkItem { OrderId = 123 }); await queue.EnqueueAsync(new OrderWorkItem { OrderId = 456 }); // Process all items await job.RunUntilEmptyAsync(); // Or run continuously await job.RunContinuousAsync(cancellationToken: stoppingToken); ``` ## Work Item Jobs Work item jobs run in a shared pool and are triggered by messages on the message bus: ### Define a Work Item Handler ```csharp using Foundatio.Jobs; public class DeleteEntityWorkItemHandler : WorkItemHandlerBase { private readonly IEntityService _entityService; private readonly ILogger _logger; public DeleteEntityWorkItemHandler( IEntityService entityService, ILogger logger) { _entityService = entityService; _logger = logger; } public override async Task HandleItemAsync(WorkItemContext ctx) { var workItem = ctx.GetData(); await ctx.ReportProgressAsync(0, "Starting deletion..."); // Delete entity and all children var children = await _entityService.GetChildrenAsync(workItem.EntityId); var total = children.Count; var current = 0; foreach (var child in children) { await _entityService.DeleteAsync(child.Id); current++; await ctx.ReportProgressAsync( (current * 100) / total, $"Deleted {current} of {total} children" ); } await _entityService.DeleteAsync(workItem.EntityId); await ctx.ReportProgressAsync(100, "Deletion complete"); } } public record DeleteEntityWorkItem { public int EntityId { get; init; } } ``` ### Register and Run Work Item Jobs ```csharp // Register handlers var handlers = new WorkItemHandlers(); handlers.Register(); // Register with DI services.AddSingleton(handlers); services.AddSingleton>(sp => new InMemoryQueue()); services.AddScoped(); // Run the job pool var job = serviceProvider.GetRequiredService(); await new JobRunner(job, instanceCount: 2).RunAsync(stoppingToken); ``` ### Trigger Work Items ```csharp // Enqueue work item var queue = serviceProvider.GetRequiredService>(); await queue.EnqueueAsync(new DeleteEntityWorkItem { EntityId = 123 }); // Subscribe to progress updates var messageBus = serviceProvider.GetRequiredService(); await messageBus.SubscribeAsync(status => { Console.WriteLine($"Progress: {status.Progress}% - {status.Message}"); }); ``` ## Job Runner Run jobs with various configurations: ```csharp using Foundatio.Jobs; var job = new CleanupJob(logger); var runner = new JobRunner(job); // Run until cancelled await runner.RunAsync(stoppingToken); // Run in background runner.RunInBackground(); // Multiple instances var multiRunner = new JobRunner(job, instanceCount: 4); await multiRunner.RunAsync(stoppingToken); ``` ## Job Options Configure job behavior: ```csharp public class MyJob : JobBase { protected override JobOptions GetDefaultOptions() { return new JobOptions { Name = "MyJob", Interval = TimeSpan.FromMinutes(5), IterationLimit = -1 // No limit }; } } ``` ### Using Job Options ```csharp var options = new JobOptions { Name = "CleanupJob", Interval = TimeSpan.FromHours(1), IterationLimit = 100 }; await job.RunContinuousAsync(options, stoppingToken); ``` ## Hosted Service Integration Run jobs as ASP.NET Core hosted services: ```csharp public class CleanupJobHostedService : BackgroundService { private readonly IServiceProvider _services; private readonly ILogger _logger; public CleanupJobHostedService( IServiceProvider services, ILogger logger) { _services = services; _logger = logger; } protected override async Task ExecuteAsync(CancellationToken stoppingToken) { _logger.LogInformation("Cleanup job starting"); while (!stoppingToken.IsCancellationRequested) { using var scope = _services.CreateScope(); var job = scope.ServiceProvider.GetRequiredService(); try { await job.RunAsync(stoppingToken); } catch (Exception ex) { _logger.LogError(ex, "Cleanup job failed"); } await Task.Delay(TimeSpan.FromHours(1), stoppingToken); } } } // Register services.AddScoped(); services.AddHostedService(); ``` ## Common Patterns ### Scheduled Jobs Run jobs on a schedule using Foundatio.Extensions.Hosting: ```csharp using Foundatio.Extensions.Hosting.Jobs; services.AddCronJob("0 */6 * * *"); // Every 6 hours services.AddCronJob("0 0 * * MON"); // Every Monday at midnight ``` ### Job with Locking Ensure only one instance runs: ```csharp public class SingletonJob : JobBase { private readonly ILockProvider _locker; public SingletonJob(ILockProvider locker) { _locker = locker; } protected override async Task RunInternalAsync(JobContext context) { await using var @lock = await _locker.AcquireAsync("singleton-job"); if (@lock == null) { _logger.LogDebug("Another instance is running"); return JobResult.Success; } // Only one instance runs this return await DoWorkAsync(context.CancellationToken); } } ``` ### Job with Progress Reporting ```csharp public class ImportJob : JobBase { private readonly IMessageBus _messageBus; protected override async Task RunInternalAsync(JobContext context) { var items = await GetItemsToImportAsync(); var total = items.Count; var processed = 0; foreach (var item in items) { if (context.CancellationToken.IsCancellationRequested) return JobResult.Cancelled; await ImportItemAsync(item); processed++; await _messageBus.PublishAsync(new ImportProgress { ProcessedCount = processed, TotalCount = total, PercentComplete = (processed * 100) / total }); } return JobResult.Success; } } ``` ### Retry Failed Jobs ```csharp public class RetryableJob : JobBase { private readonly ResiliencePolicy _policy; public RetryableJob() { _policy = new ResiliencePolicyBuilder() .WithMaxAttempts(3) .WithExponentialDelay(TimeSpan.FromSeconds(1)) .Build(); } protected override async Task RunInternalAsync(JobContext context) { try { await _policy.ExecuteAsync(async ct => { await DoUnreliableWorkAsync(ct); }, context.CancellationToken); return JobResult.Success; } catch (Exception ex) { return JobResult.FromException(ex); } } } ``` ## Dependency Injection ### Register Jobs ```csharp services.AddScoped(); services.AddScoped(); services.AddSingleton>(sp => new InMemoryQueue()); ``` ### Queue Jobs with DI ```csharp services.AddSingleton>(sp => new InMemoryQueue() ); services.AddScoped(); services.AddHostedService>(); ``` ## Best Practices ### 1. Use Cancellation Tokens ```csharp protected override async Task RunInternalAsync(JobContext context) { foreach (var item in items) { // Check for cancellation context.CancellationToken.ThrowIfCancellationRequested(); await ProcessItemAsync(item, context.CancellationToken); } return JobResult.Success; } ``` ### 2. Log Job Progress ```csharp protected override async Task RunInternalAsync(JobContext context) { using var _ = _logger.BeginScope(new { JobRunId = Guid.NewGuid() }); _logger.LogInformation("Starting job"); try { await DoWorkAsync(); _logger.LogInformation("Job completed successfully"); return JobResult.Success; } catch (Exception ex) { _logger.LogError(ex, "Job failed"); return JobResult.FromException(ex); } } ``` ### 3. Keep Jobs Idempotent ```csharp protected override async Task RunInternalAsync(JobContext context) { // Track what was processed var lastProcessedId = await _state.GetLastProcessedIdAsync(); var items = await _db.GetItemsAfterAsync(lastProcessedId); foreach (var item in items) { await ProcessItemAsync(item); await _state.SetLastProcessedIdAsync(item.Id); } return JobResult.Success; } ``` ### 4. Handle Transient Failures ```csharp protected override async Task RunInternalAsync(JobContext context) { try { await DoWorkAsync(context.CancellationToken); return JobResult.Success; } catch (TransientException ex) { // Fail and allow retry return JobResult.FailedWithMessage(ex.Message); } catch (PermanentException ex) { // Log and succeed to prevent retries _logger.LogError(ex, "Permanent failure - not retrying"); return JobResult.Success; } } ``` ### 5. Use Appropriate Job Type | Use Case | Job Type | |----------|----------| | Scheduled maintenance | Standard Job | | Process queue items | Queue Processor Job | | On-demand heavy tasks | Work Item Job | | Event-driven processing | Work Item Job | ## Next Steps * [Queues](./queues) - Queue implementations for job processing * [Locks](./locks) - Distributed locking for singleton jobs * [Resilience](./resilience) - Retry policies for job reliability --- --- url: /guide/locks.md --- # Locks Locks ensure a resource is only accessed by one consumer at any given time. Foundatio provides distributed locking implementations through the `ILockProvider` interface. ## The ILockProvider Interface ```csharp public interface ILockProvider { Task AcquireAsync(string resource, TimeSpan? timeUntilExpires = null, bool releaseOnDispose = true, CancellationToken cancellationToken = default); Task IsLockedAsync(string resource); Task ReleaseAsync(string resource, string lockId); Task ReleaseAsync(string resource); Task RenewAsync(string resource, string lockId, TimeSpan? timeUntilExpires = null); } public interface ILock : IAsyncDisposable { Task RenewAsync(TimeSpan? timeUntilExpires = null); Task ReleaseAsync(); string LockId { get; } string Resource { get; } DateTime AcquiredTimeUtc { get; } TimeSpan TimeWaitedForLock { get; } int RenewalCount { get; } } ``` ## Implementations ### CacheLockProvider Uses a cache client and message bus for distributed locking: ```csharp using Foundatio.Lock; using Foundatio.Caching; using Foundatio.Messaging; var cache = new InMemoryCacheClient(); var messageBus = new InMemoryMessageBus(); var locker = new CacheLockProvider(cache, messageBus); await using var @lock = await locker.AcquireAsync("my-resource"); if (@lock != null) { // Exclusive access to resource await DoExclusiveWorkAsync(); } ``` With Redis for production: ```csharp var redis = await ConnectionMultiplexer.ConnectAsync("localhost:6379"); var cache = new RedisCacheClient(o => o.ConnectionMultiplexer = redis); var messageBus = new RedisMessageBus(o => o.Subscriber = redis.GetSubscriber()); var locker = new CacheLockProvider(cache, messageBus); ``` ### ThrottlingLockProvider Limits the number of operations within a time period: ```csharp using Foundatio.Lock; var throttledLocker = new ThrottlingLockProvider( cache, maxHits: 10, // Maximum locks allowed period: TimeSpan.FromMinutes(1) // Per time period ); // Only allows 10 operations per minute across all instances var @lock = await throttledLocker.AcquireAsync("api-rate-limit"); if (@lock != null) { await CallExternalApiAsync(); await @lock.ReleaseAsync(); } else { // Rate limited throw new TooManyRequestsException(); } ``` ### ScopedLockProvider Prefixes all lock keys with a scope: ```csharp using Foundatio.Lock; var baseLock = new CacheLockProvider(cache, messageBus); var tenantLock = new ScopedLockProvider(baseLock, "tenant:abc"); // Lock key becomes: "tenant:abc:resource-1" await using var @lock = await tenantLock.AcquireAsync("resource-1"); ``` ## Basic Usage ### Acquire and Release ```csharp var locker = new CacheLockProvider(cache, messageBus); // Acquire lock var @lock = await locker.AcquireAsync("my-resource"); if (@lock != null) { try { // Do exclusive work await ProcessAsync(); } finally { // Always release await @lock.ReleaseAsync(); } } ``` ### Using Dispose Pattern The recommended pattern uses `await using` for automatic release: ```csharp await using var @lock = await locker.AcquireAsync("my-resource"); if (@lock != null) { // Lock is automatically released when scope ends await DoExclusiveWorkAsync(); } ``` ### Non-Blocking Acquire Check if lock was acquired: ```csharp await using var @lock = await locker.AcquireAsync("my-resource"); if (@lock == null) { // Resource is locked by another process return; } // Got the lock await DoWorkAsync(); ``` ### Blocking Acquire with Timeout Wait for lock with cancellation: ```csharp using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(30)); try { await using var @lock = await locker.AcquireAsync( "my-resource", cancellationToken: cts.Token ); if (@lock != null) { await DoWorkAsync(); } } catch (OperationCanceledException) { // Timeout waiting for lock _logger.LogWarning("Timed out waiting for lock"); } ``` ## Lock Expiration ### Setting Expiration Locks expire automatically to prevent deadlocks: ```csharp // Lock expires after 5 minutes await using var @lock = await locker.AcquireAsync( "my-resource", timeUntilExpires: TimeSpan.FromMinutes(5) ); ``` ### Renewing Locks For long-running operations, renew the lock: ```csharp await using var @lock = await locker.AcquireAsync( "my-resource", timeUntilExpires: TimeSpan.FromMinutes(1) ); if (@lock != null) { // Do some work await DoPartOneAsync(); // Renew lock for more time await @lock.RenewAsync(TimeSpan.FromMinutes(1)); // Continue work await DoPartTwoAsync(); } ``` ### Automatic Renewal For very long operations, set up automatic renewal: ```csharp await using var @lock = await locker.AcquireAsync("my-resource"); if (@lock == null) return; using var cts = new CancellationTokenSource(); // Start renewal task var renewTask = Task.Run(async () => { while (!cts.Token.IsCancellationRequested) { await Task.Delay(TimeSpan.FromSeconds(30), cts.Token); await @lock.RenewAsync(TimeSpan.FromMinutes(1)); } }); try { await VeryLongRunningOperationAsync(); } finally { cts.Cancel(); } ``` ## Common Patterns ### Singleton Processing Ensure only one instance processes a resource: ```csharp public async Task ProcessOrderAsync(int orderId) { await using var @lock = await _locker.AcquireAsync($"order:{orderId}"); if (@lock == null) { _logger.LogInformation("Order {OrderId} is being processed elsewhere", orderId); return; } // Only one instance processes this order await DoProcessingAsync(orderId); } ``` ### Leader Election Elect a single instance for a job: ```csharp public async Task RunAsLeaderAsync(CancellationToken ct) { while (!ct.IsCancellationRequested) { await using var @lock = await _locker.AcquireAsync("leader:job-runner"); if (@lock != null) { _logger.LogInformation("This instance is now the leader"); // Keep renewing while leading while (!ct.IsCancellationRequested) { await DoLeaderWorkAsync(); await @lock.RenewAsync(); await Task.Delay(TimeSpan.FromSeconds(5), ct); } } else { // Not leader, wait and try again await Task.Delay(TimeSpan.FromSeconds(30), ct); } } } ``` ### Preventing Duplicate Operations ```csharp public async Task CreateOrderAsync(CreateOrderRequest request) { // Prevent duplicate orders for same customer await using var @lock = await _locker.AcquireAsync( $"create-order:{request.CustomerId}", timeUntilExpires: TimeSpan.FromSeconds(30) ); if (@lock == null) { throw new ConcurrencyException("Another order is being created"); } // Check for recent duplicates var recentOrder = await _db.GetRecentOrderAsync(request.CustomerId); if (recentOrder != null && recentOrder.IsSimilar(request)) { throw new DuplicateOrderException(); } return await _db.CreateOrderAsync(request); } ``` ### Using TryUsingAsync Extension Simplified pattern for lock-protected operations: ```csharp var success = await locker.TryUsingAsync( "my-resource", async ct => { await DoExclusiveWorkAsync(ct); }, timeUntilExpires: TimeSpan.FromMinutes(5), cancellationToken ); if (!success) { _logger.LogWarning("Could not acquire lock"); } ``` ## Rate Limiting with ThrottlingLockProvider ### API Rate Limiting ```csharp public class RateLimitedApiClient { private readonly ThrottlingLockProvider _throttler; private readonly HttpClient _client; public RateLimitedApiClient(ICacheClient cache, HttpClient client) { _throttler = new ThrottlingLockProvider( cache, maxHits: 100, // 100 requests period: TimeSpan.FromMinutes(1) // per minute ); _client = client; } public async Task GetAsync(string endpoint) { await using var @lock = await _throttler.AcquireAsync("external-api"); if (@lock == null) { throw new RateLimitExceededException(); } var response = await _client.GetAsync(endpoint); return await response.Content.ReadFromJsonAsync(); } } ``` ### Per-User Rate Limiting ```csharp public async Task ProcessRequest(string userId) { // 10 requests per minute per user await using var @lock = await _throttler.AcquireAsync($"user:{userId}:api"); if (@lock == null) { return StatusCode(429, "Too many requests"); } return Ok(await ProcessAsync()); } ``` ## Dependency Injection ### Basic Registration ```csharp services.AddSingleton(); services.AddSingleton(); services.AddSingleton(sp => new CacheLockProvider( sp.GetRequiredService(), sp.GetRequiredService() ) ); ``` ### With Redis ```csharp services.AddSingleton( await ConnectionMultiplexer.ConnectAsync("localhost:6379") ); services.AddSingleton(sp => new RedisCacheClient(o => o.ConnectionMultiplexer = sp.GetRequiredService() ) ); services.AddSingleton(sp => new RedisMessageBus(o => o.Subscriber = sp.GetRequiredService().GetSubscriber() ) ); services.AddSingleton(sp => new CacheLockProvider( sp.GetRequiredService(), sp.GetRequiredService() ) ); ``` ### Multiple Lock Providers ```csharp // General-purpose locking services.AddKeyedSingleton("general", (sp, _) => new CacheLockProvider( sp.GetRequiredService(), sp.GetRequiredService() ) ); // Rate limiting services.AddKeyedSingleton("throttle", (sp, _) => new ThrottlingLockProvider( sp.GetRequiredService(), maxHits: 100, period: TimeSpan.FromMinutes(1) ) ); ``` ## Best Practices ### 1. Always Handle Null Lock ```csharp // ✅ Good: Check for null await using var @lock = await locker.AcquireAsync("resource"); if (@lock == null) { return; // or throw, or retry } // ❌ Bad: Assume lock acquired await using var @lock = await locker.AcquireAsync("resource"); await DoWork(); // May not have the lock! ``` ### 2. Use Meaningful Lock Names ```csharp // ✅ Good: Descriptive, hierarchical await locker.AcquireAsync($"order:process:{orderId}"); await locker.AcquireAsync($"user:{userId}:balance:update"); // ❌ Bad: Generic, ambiguous await locker.AcquireAsync("lock1"); await locker.AcquireAsync("resource"); ``` ### 3. Set Appropriate Expiration ```csharp // Match expiration to expected operation duration await locker.AcquireAsync("quick-op", TimeSpan.FromSeconds(10)); await locker.AcquireAsync("long-op", TimeSpan.FromMinutes(5)); ``` ### 4. Prefer `await using` Pattern ```csharp // ✅ Good: Automatic release await using var @lock = await locker.AcquireAsync("resource"); // ⚠️ Be careful: Manual release required var @lock = await locker.AcquireAsync("resource", releaseOnDispose: false); try { await DoWork(); } finally { await @lock.ReleaseAsync(); } ``` ### 5. Use Scoped Locks for Multi-Tenant ```csharp var tenantLock = new ScopedLockProvider(baseLock, $"tenant:{tenantId}"); // All locks are isolated per tenant ``` ## Lock Information Access lock metadata: ```csharp await using var @lock = await locker.AcquireAsync("resource"); if (@lock != null) { Console.WriteLine($"Lock ID: {@lock.LockId}"); Console.WriteLine($"Resource: {@lock.Resource}"); Console.WriteLine($"Acquired: {@lock.AcquiredTimeUtc}"); Console.WriteLine($"Wait time: {@lock.TimeWaitedForLock}"); Console.WriteLine($"Renewals: {@lock.RenewalCount}"); } ``` ## Next Steps * [Caching](./caching) - Cache implementations used by locks * [Messaging](./messaging) - Message bus used for cache invalidation * [Resilience](./resilience) - Retry policies for lock acquisition --- --- url: /guide/messaging.md --- # Messaging Messaging allows you to publish and subscribe to messages flowing through your application using pub/sub patterns. Foundatio provides multiple message bus implementations through the `IMessageBus` interface. ## The IMessageBus Interface ```csharp public interface IMessageBus : IMessagePublisher, IMessageSubscriber, IDisposable { } public interface IMessagePublisher { Task PublishAsync(Type messageType, object message, MessageOptions options = null, CancellationToken cancellationToken = default); } public interface IMessageSubscriber { Task SubscribeAsync(Func handler, CancellationToken cancellationToken = default) where T : class; } ``` ## Implementations ### InMemoryMessageBus An in-memory message bus for development and testing: ```csharp using Foundatio.Messaging; var messageBus = new InMemoryMessageBus(); // Subscribe to messages await messageBus.SubscribeAsync(async msg => { Console.WriteLine($"Order created: {msg.OrderId}"); }); // Publish a message await messageBus.PublishAsync(new OrderCreated { OrderId = 123 }); ``` ### RedisMessageBus Distributed messaging using Redis pub/sub (separate package): ```csharp // dotnet add package Foundatio.Redis using Foundatio.Redis.Messaging; using StackExchange.Redis; var redis = await ConnectionMultiplexer.ConnectAsync("localhost:6379"); var messageBus = new RedisMessageBus(o => o.Subscriber = redis.GetSubscriber()); ``` ### RabbitMQMessageBus Messaging using RabbitMQ (separate package): ```csharp // dotnet add package Foundatio.RabbitMQ using Foundatio.RabbitMQ.Messaging; var messageBus = new RabbitMQMessageBus(o => { o.ConnectionString = "amqp://guest:guest@localhost:5672"; }); ``` ### KafkaMessageBus Messaging using Apache Kafka (separate package): ```csharp // dotnet add package Foundatio.Kafka using Foundatio.Kafka.Messaging; var messageBus = new KafkaMessageBus(o => { o.BootstrapServers = "localhost:9092"; }); ``` ### AzureServiceBusMessageBus Messaging using Azure Service Bus (separate package): ```csharp // dotnet add package Foundatio.AzureServiceBus using Foundatio.AzureServiceBus.Messaging; var messageBus = new AzureServiceBusMessageBus(o => { o.ConnectionString = "..."; o.Topic = "events"; }); ``` ## Basic Usage ### Publishing Messages ```csharp var messageBus = new InMemoryMessageBus(); // Simple publish await messageBus.PublishAsync(new OrderCreated { OrderId = 123 }); // With options await messageBus.PublishAsync(new OrderCreated { OrderId = 123 }, new MessageOptions { CorrelationId = "request-abc", DeliveryDelay = TimeSpan.FromSeconds(30), Properties = new Dictionary { ["source"] = "order-service" } }); // Delayed publish (extension method) await messageBus.PublishAsync( new OrderReminder { OrderId = 123 }, TimeSpan.FromHours(1) ); ``` ### Subscribing to Messages ```csharp var messageBus = new InMemoryMessageBus(); // Simple subscription await messageBus.SubscribeAsync(async order => { Console.WriteLine($"Processing order: {order.OrderId}"); }); // With cancellation token await messageBus.SubscribeAsync( async (order, ct) => { await ProcessOrderAsync(order, ct); }, cancellationToken ); // Synchronous handler await messageBus.SubscribeAsync(order => { Console.WriteLine($"Order: {order.OrderId}"); }); ``` ### Multiple Subscribers Each subscriber receives every message: ```csharp var messageBus = new InMemoryMessageBus(); // Handler 1: Logging await messageBus.SubscribeAsync(async order => { _logger.LogInformation("Order {OrderId} created", order.OrderId); }); // Handler 2: Notification await messageBus.SubscribeAsync(async order => { await _notificationService.SendAsync(order.CustomerId, "Order placed!"); }); // Handler 3: Analytics await messageBus.SubscribeAsync(async order => { await _analytics.TrackAsync("order_created", order.OrderId); }); // All three handlers receive this message await messageBus.PublishAsync(new OrderCreated { OrderId = 123 }); ``` ## Message Types ### Define Your Messages ```csharp // Simple message public record OrderCreated { public int OrderId { get; init; } public DateTime CreatedAt { get; init; } public string CustomerId { get; init; } } // Message with interface for grouping public interface IOrderEvent { int OrderId { get; } } public record OrderShipped : IOrderEvent { public int OrderId { get; init; } public string TrackingNumber { get; init; } } public record OrderDelivered : IOrderEvent { public int OrderId { get; init; } public DateTime DeliveredAt { get; init; } } ``` ### Subscribe to Interface Subscribe to all messages implementing an interface: ```csharp // Receives OrderShipped, OrderDelivered, and any other IOrderEvent await messageBus.SubscribeAsync(async orderEvent => { _logger.LogInformation("Order event: {Type} for {OrderId}", orderEvent.GetType().Name, orderEvent.OrderId); }); ``` ### IMessage Interface Use the built-in `IMessage` interface for raw message access: ```csharp await messageBus.SubscribeAsync(async (IMessage message, CancellationToken ct) => { Console.WriteLine($"Type: {message.Type}"); Console.WriteLine($"Correlation ID: {message.CorrelationId}"); // Deserialize the data var order = message.GetBody(); }); ``` ## Common Patterns ### Event-Driven Architecture Decouple services with events: ```csharp // Order Service public class OrderService { private readonly IMessageBus _messageBus; public async Task CreateOrderAsync(CreateOrderRequest request) { var order = await _repository.CreateAsync(request); // Publish event for other services await _messageBus.PublishAsync(new OrderCreated { OrderId = order.Id, CustomerId = request.CustomerId, CreatedAt = DateTime.UtcNow }); } } // Inventory Service (separate process/service) public class InventoryService { public InventoryService(IMessageBus messageBus) { messageBus.SubscribeAsync(async order => { await ReserveInventoryAsync(order.OrderId); }); } } // Notification Service (separate process/service) public class NotificationService { public NotificationService(IMessageBus messageBus) { messageBus.SubscribeAsync(async order => { await SendConfirmationEmailAsync(order.CustomerId); }); } } ``` ### Cache Invalidation Coordinate cache across instances: ```csharp public class CacheInvalidationService { private readonly IMessageBus _messageBus; private readonly ICacheClient _localCache; public CacheInvalidationService(IMessageBus messageBus, ICacheClient localCache) { _messageBus = messageBus; _localCache = localCache; // Listen for invalidation messages _messageBus.SubscribeAsync(async msg => { await _localCache.RemoveAsync(msg.Key); }); } public async Task InvalidateAsync(string key) { // Remove locally await _localCache.RemoveAsync(key); // Notify other instances await _messageBus.PublishAsync(new CacheInvalidated { Key = key }); } } public record CacheInvalidated { public string Key { get; init; } } ``` ### Real-Time Updates Push updates to clients: ```csharp // Server-side public class NotificationHub { private readonly IMessageBus _messageBus; public NotificationHub(IMessageBus messageBus) { _messageBus = messageBus; // Forward bus messages to SignalR/WebSocket _messageBus.SubscribeAsync(async notification => { await _hubContext.Clients .User(notification.UserId) .SendAsync("notification", notification); }); } } // When something happens await messageBus.PublishAsync(new UserNotification { UserId = "user-123", Message = "Your order has shipped!" }); ``` ### Saga/Process Manager Coordinate multi-step processes: ```csharp public class OrderSaga { private readonly IMessageBus _messageBus; public OrderSaga(IMessageBus messageBus) { _messageBus = messageBus; // Step 1: Order created -> Reserve inventory _messageBus.SubscribeAsync(async order => { await ReserveInventoryAsync(order.OrderId); await _messageBus.PublishAsync(new InventoryReserved { OrderId = order.OrderId }); }); // Step 2: Inventory reserved -> Process payment _messageBus.SubscribeAsync(async evt => { await ProcessPaymentAsync(evt.OrderId); await _messageBus.PublishAsync(new PaymentProcessed { OrderId = evt.OrderId }); }); // Step 3: Payment processed -> Ship order _messageBus.SubscribeAsync(async evt => { await ShipOrderAsync(evt.OrderId); }); } } ``` ## Message Options Configure message delivery: ```csharp await messageBus.PublishAsync(new OrderCreated { OrderId = 123 }, new MessageOptions { // Unique message identifier UniqueId = Guid.NewGuid().ToString(), // For tracing across services CorrelationId = Activity.Current?.Id, // Delayed delivery DeliveryDelay = TimeSpan.FromMinutes(5), // Custom properties Properties = new Dictionary { ["source"] = "order-service", ["version"] = "1.0" } }); ``` ## Dependency Injection ### Basic Registration ```csharp // In-memory (development) services.AddSingleton(); // Redis (production) services.AddSingleton(sp => { var redis = sp.GetRequiredService(); return new RedisMessageBus(o => o.Subscriber = redis.GetSubscriber()); }); ``` ### Subscribe at Startup ```csharp public class MessageSubscriber : IHostedService { private readonly IMessageBus _messageBus; private readonly IServiceProvider _services; public MessageSubscriber(IMessageBus messageBus, IServiceProvider services) { _messageBus = messageBus; _services = services; } public async Task StartAsync(CancellationToken cancellationToken) { await _messageBus.SubscribeAsync(async (msg, ct) => { using var scope = _services.CreateScope(); var handler = scope.ServiceProvider.GetRequiredService(); await handler.HandleAsync(msg, ct); }, cancellationToken); } public Task StopAsync(CancellationToken cancellationToken) => Task.CompletedTask; } // Register services.AddHostedService(); ``` ## Error Handling ### In Subscribers ```csharp await messageBus.SubscribeAsync(async order => { try { await ProcessOrderAsync(order); } catch (Exception ex) { _logger.LogError(ex, "Failed to process order {OrderId}", order.OrderId); // Optionally publish failure event await _messageBus.PublishAsync(new OrderProcessingFailed { OrderId = order.OrderId, Error = ex.Message }); } }); ``` ### With Retry ```csharp await messageBus.SubscribeAsync(async order => { await _resiliencePolicy.ExecuteAsync(async ct => { await ProcessOrderAsync(order, ct); }); }); ``` ## Best Practices ### 1. Use Immutable Messages ```csharp // ✅ Good: Immutable record public record OrderCreated { public int OrderId { get; init; } public required string CustomerId { get; init; } } // ❌ Bad: Mutable class public class OrderCreated { public int OrderId { get; set; } public string CustomerId { get; set; } } ``` ### 2. Include Timestamp and Correlation ```csharp public record OrderCreated { public int OrderId { get; init; } public DateTime OccurredAt { get; init; } = DateTime.UtcNow; public string CorrelationId { get; init; } = Activity.Current?.Id; } ``` ### 3. Handle Idempotency ```csharp await messageBus.SubscribeAsync(async order => { // Check if already processed if (await _processedEvents.ContainsAsync(order.EventId)) { _logger.LogDebug("Already processed {EventId}", order.EventId); return; } await ProcessOrderAsync(order); await _processedEvents.AddAsync(order.EventId); }); ``` ### 4. Use Specific Message Types ```csharp // ✅ Good: Specific, intentional messages public record OrderCreated { ... } public record OrderShipped { ... } public record OrderCancelled { ... } // ❌ Bad: Generic, multi-purpose messages public record OrderEvent { public string Action { get; set; } } ``` ### 5. Keep Messages Small ```csharp // ✅ Good: Just identifiers public record OrderCreated { public int OrderId { get; init; } } // ❌ Bad: Full entity in message public record OrderCreated { public Order FullOrderWithAllDetails { get; init; } } ``` ## Next Steps * [Queues](./queues) - For guaranteed delivery with acknowledgment * [Caching](./caching) - Cache invalidation with messaging * [Jobs](./jobs) - Background processing triggered by messages --- --- url: /guide/queues.md --- # Queues Queues offer First In, First Out (FIFO) message delivery with reliable processing semantics. Foundatio provides multiple queue implementations through the `IQueue` interface. ## The IQueue Interface ```csharp public interface IQueue : IQueue where T : class { AsyncEvent> Enqueuing { get; } AsyncEvent> Enqueued { get; } AsyncEvent> Dequeued { get; } AsyncEvent> LockRenewed { get; } AsyncEvent> Completed { get; } AsyncEvent> Abandoned { get; } void AttachBehavior(IQueueBehavior behavior); Task EnqueueAsync(T data, QueueEntryOptions options = null); Task> DequeueAsync(CancellationToken cancellationToken); Task> DequeueAsync(TimeSpan? timeout = null); Task RenewLockAsync(IQueueEntry queueEntry); Task CompleteAsync(IQueueEntry queueEntry); Task AbandonAsync(IQueueEntry queueEntry); Task> GetDeadletterItemsAsync(CancellationToken cancellationToken = default); Task StartWorkingAsync(Func, CancellationToken, Task> handler, bool autoComplete = false, CancellationToken cancellationToken = default); } public interface IQueue : IHaveSerializer, IDisposable { Task GetQueueStatsAsync(); Task DeleteQueueAsync(); string QueueId { get; } } ``` ## Implementations ### InMemoryQueue An in-memory queue implementation for development and testing: ```csharp using Foundatio.Queues; var queue = new InMemoryQueue(); // Enqueue work await queue.EnqueueAsync(new WorkItem { Id = 1, Data = "Hello" }); // Dequeue and process var entry = await queue.DequeueAsync(); Console.WriteLine($"Processing: {entry.Value.Data}"); await entry.CompleteAsync(); ``` ### RedisQueue Distributed queue using Redis (separate package): ```csharp // dotnet add package Foundatio.Redis using Foundatio.Redis.Queues; var queue = new RedisQueue(o => { o.ConnectionMultiplexer = redis; o.Name = "work-items"; o.WorkItemTimeout = TimeSpan.FromMinutes(5); }); ``` ### AzureServiceBusQueue Queue using Azure Service Bus (separate package): ```csharp // dotnet add package Foundatio.AzureServiceBus using Foundatio.AzureServiceBus.Queues; var queue = new AzureServiceBusQueue(o => { o.ConnectionString = "..."; o.Name = "work-items"; }); ``` ### AzureStorageQueue Queue using Azure Storage Queues (separate package): ```csharp // dotnet add package Foundatio.AzureStorage using Foundatio.AzureStorage.Queues; var queue = new AzureStorageQueue(o => { o.ConnectionString = "..."; o.Name = "work-items"; }); ``` ### SQSQueue Queue using AWS SQS (separate package): ```csharp // dotnet add package Foundatio.AWS using Foundatio.AWS.Queues; var queue = new SQSQueue(o => { o.Region = RegionEndpoint.USEast1; o.QueueName = "work-items"; }); ``` ## Queue Entry Lifecycle Each dequeued message goes through a lifecycle: ```mermaid graph LR A[Queued] --> B[Dequeued/Working] B --> C{Processing} C -->|Success| D[Completed] C -->|Failure| E[Abandoned] E --> F{Retry?} F -->|Yes| A F -->|No| G[Dead Letter] ``` ### Completing Entries Mark an entry as successfully processed: ```csharp var entry = await queue.DequeueAsync(); try { await ProcessAsync(entry.Value); await entry.CompleteAsync(); } catch { await entry.AbandonAsync(); throw; } ``` ### Abandoning Entries Return an entry to the queue for retry: ```csharp var entry = await queue.DequeueAsync(); if (!CanProcess(entry.Value)) { // Return to queue for later processing await entry.AbandonAsync(); return; } ``` ### Lock Renewal For long-running operations, renew the lock: ```csharp var entry = await queue.DequeueAsync(); using var cts = new CancellationTokenSource(); // Renew lock periodically var renewTask = Task.Run(async () => { while (!cts.Token.IsCancellationRequested) { await Task.Delay(TimeSpan.FromSeconds(30), cts.Token); await entry.RenewLockAsync(); } }); try { await LongRunningProcessAsync(entry.Value); await entry.CompleteAsync(); } finally { cts.Cancel(); } ``` ## Processing Patterns ### Simple Processing Loop ```csharp while (!cancellationToken.IsCancellationRequested) { var entry = await queue.DequeueAsync(cancellationToken); if (entry == null) continue; try { await ProcessAsync(entry.Value); await entry.CompleteAsync(); } catch (Exception ex) { _logger.LogError(ex, "Failed to process {Id}", entry.Value.Id); await entry.AbandonAsync(); } } ``` ### Using StartWorkingAsync Simplified background processing: ```csharp // Start processing in background await queue.StartWorkingAsync( async (entry, ct) => { await ProcessAsync(entry.Value); }, autoComplete: true, // Automatically complete on success cancellationToken ); ``` ### Parallel Processing Process multiple items concurrently: ```csharp var semaphore = new SemaphoreSlim(maxConcurrency); while (!cancellationToken.IsCancellationRequested) { await semaphore.WaitAsync(cancellationToken); _ = Task.Run(async () => { try { var entry = await queue.DequeueAsync(cancellationToken); if (entry != null) { await ProcessAsync(entry.Value); await entry.CompleteAsync(); } } finally { semaphore.Release(); } }); } ``` ## Queue Entry Options Configure enqueue behavior: ```csharp await queue.EnqueueAsync(new WorkItem { Id = 1 }, new QueueEntryOptions { UniqueId = "unique-id", // Dedupe by ID CorrelationId = "request-123", // For tracing DeliveryDelay = TimeSpan.FromMinutes(5), // Delayed delivery Properties = new Dictionary { ["priority"] = "high" } }); ``` ## Queue Events Subscribe to queue lifecycle events: ```csharp var queue = new InMemoryQueue(); queue.Enqueuing.AddHandler(async (sender, args) => { _logger.LogInformation("Enqueuing: {Data}", args.Entry.Value); }); queue.Enqueued.AddHandler(async (sender, args) => { _logger.LogInformation("Enqueued: {Id}", args.Entry.Id); }); queue.Dequeued.AddHandler(async (sender, args) => { _logger.LogInformation("Dequeued: {Id}", args.Entry.Id); }); queue.Completed.AddHandler(async (sender, args) => { _logger.LogInformation("Completed: {Id}", args.Entry.Id); }); queue.Abandoned.AddHandler(async (sender, args) => { _logger.LogWarning("Abandoned: {Id}", args.Entry.Id); }); ``` ## Queue Behaviors Extend queue functionality with behaviors: ```csharp public class LoggingQueueBehavior : IQueueBehavior where T : class { private readonly ILogger _logger; public LoggingQueueBehavior(ILogger logger) => _logger = logger; public void Attach(IQueue queue) { queue.Enqueued.AddHandler(async (s, e) => _logger.LogInformation("Enqueued {Id}", e.Entry.Id)); queue.Completed.AddHandler(async (s, e) => _logger.LogInformation("Completed {Id}", e.Entry.Id)); } } // Attach to queue queue.AttachBehavior(new LoggingQueueBehavior(logger)); ``` ## Queue Statistics Monitor queue health: ```csharp var stats = await queue.GetQueueStatsAsync(); Console.WriteLine($"Queued: {stats.Queued}"); Console.WriteLine($"Working: {stats.Working}"); Console.WriteLine($"Dead Letter: {stats.Deadletter}"); Console.WriteLine($"Enqueued: {stats.Enqueued}"); Console.WriteLine($"Dequeued: {stats.Dequeued}"); Console.WriteLine($"Completed: {stats.Completed}"); Console.WriteLine($"Abandoned: {stats.Abandoned}"); Console.WriteLine($"Errors: {stats.Errors}"); Console.WriteLine($"Timeouts: {stats.Timeouts}"); ``` ## Dead Letter Queue Handle failed messages: ```csharp // Get dead letter items var deadLetters = await queue.GetDeadletterItemsAsync(); foreach (var item in deadLetters) { _logger.LogWarning("Dead letter: {Id}", item.Id); // Optionally re-queue for retry await queue.EnqueueAsync(item); } ``` ## Dependency Injection ### Basic Registration ```csharp // In-memory (development) services.AddSingleton>(sp => new InMemoryQueue()); // Redis (production) services.AddSingleton>(sp => new RedisQueue(o => { o.ConnectionMultiplexer = sp.GetRequiredService(); o.Name = "work-items"; })); ``` ### Multiple Queues ```csharp services.AddSingleton>(sp => new InMemoryQueue(o => o.Name = "orders")); services.AddSingleton>(sp => new InMemoryQueue(o => o.Name = "emails")); ``` ## Best Practices ### 1. Use Typed Messages ```csharp // ✅ Good: Typed, versioned messages public record OrderWorkItem { public int Version { get; init; } = 1; public required int OrderId { get; init; } public required DateTime CreatedAt { get; init; } } // ❌ Bad: Generic, untyped public class WorkItem { public object Data { get; set; } } ``` ### 2. Handle Idempotency ```csharp var entry = await queue.DequeueAsync(); // Check if already processed if (await _processedIds.ContainsAsync(entry.Value.Id)) { await entry.CompleteAsync(); return; } // Process await ProcessAsync(entry.Value); // Mark as processed await _processedIds.AddAsync(entry.Value.Id); await entry.CompleteAsync(); ``` ### 3. Set Appropriate Timeouts ```csharp var queue = new RedisQueue(o => { o.WorkItemTimeout = TimeSpan.FromMinutes(5); // How long to process o.RetryDelay = TimeSpan.FromSeconds(30); // Delay before retry o.RetryLimit = 3; // Max retries }); ``` ### 4. Monitor Queue Depth ```csharp var stats = await queue.GetQueueStatsAsync(); if (stats.Queued > 1000) { _logger.LogWarning("Queue depth is high: {Depth}", stats.Queued); // Consider scaling workers } ``` ### 5. Use Delayed Delivery for Scheduling ```csharp // Schedule for later await queue.EnqueueAsync(reminder, new QueueEntryOptions { DeliveryDelay = TimeSpan.FromHours(24) }); ``` ## Next Steps * [Jobs](./jobs) - Queue processor jobs for background processing * [Messaging](./messaging) - Pub/sub for event-driven patterns * [Locks](./locks) - Coordinate queue processing across instances --- --- url: /guide/implementations/redis.md --- # Redis Implementation Foundatio provides Redis implementations for caching, queues, messaging, locks, and file storage. Redis enables distributed scenarios across multiple processes and servers. ## Overview | Implementation | Interface | Package | |----------------|-----------|---------| | `RedisCacheClient` | `ICacheClient` | Foundatio.Redis | | `RedisHybridCacheClient` | `ICacheClient` | Foundatio.Redis | | `RedisQueue` | `IQueue` | Foundatio.Redis | | `RedisMessageBus` | `IMessageBus` | Foundatio.Redis | | `RedisFileStorage` | `IFileStorage` | Foundatio.Redis | | `CacheLockProvider` | `ILockProvider` | Foundatio (with Redis cache) | ## Installation ```bash dotnet add package Foundatio.Redis ``` ## Connection Setup ### Basic Connection ```csharp using StackExchange.Redis; var redis = await ConnectionMultiplexer.ConnectAsync("localhost:6379"); ``` ### Production Connection ```csharp var options = new ConfigurationOptions { EndPoints = { "redis-primary:6379", "redis-replica:6379" }, Password = "your-password", Ssl = true, AbortOnConnectFail = false, ConnectRetry = 5, ConnectTimeout = 5000, SyncTimeout = 5000, AsyncTimeout = 5000 }; var redis = await ConnectionMultiplexer.ConnectAsync(options); ``` ### Connection String Format ``` redis:6379,password=secret,ssl=true,abortConnect=false ``` ## RedisCacheClient A distributed cache backed by Redis. ### Basic Usage ```csharp using Foundatio.Caching; var cache = new RedisCacheClient(options => { options.ConnectionMultiplexer = redis; }); // Store and retrieve await cache.SetAsync("user:123", user); var cachedUser = await cache.GetAsync("user:123"); // With expiration await cache.SetAsync("session:abc", session, TimeSpan.FromHours(1)); ``` ### Configuration Options ```csharp var cache = new RedisCacheClient(options => { options.ConnectionMultiplexer = redis; options.LoggerFactory = loggerFactory; options.Serializer = new SystemTextJsonSerializer(); }); ``` ### Advanced Operations ```csharp // Increment/Decrement await cache.IncrementAsync("counter", 1); await cache.IncrementAsync("views", 1.5); // Set if not exists var added = await cache.AddAsync("lock-key", "locked", TimeSpan.FromSeconds(30)); // Batch operations await cache.SetAllAsync(new Dictionary { ["user:1"] = user1, ["user:2"] = user2 }); var users = await cache.GetAllAsync(new[] { "user:1", "user:2" }); ``` ## RedisHybridCacheClient Combines Redis with a local in-memory cache for optimal performance. ### How It Works ``` ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ Request │───▶│ Local Cache │───▶│ Redis Cache │ └─────────────┘ └─────────────┘ └─────────────┘ │ │ ▼ ▼ Cache Hit? Cache Hit? │ │ Yes: Return Yes: Return & Store Local ``` ### Basic Usage ```csharp var hybridCache = new RedisHybridCacheClient(options => { options.ConnectionMultiplexer = redis; options.LocalCacheMaxItems = 1000; options.LoggerFactory = loggerFactory; }); ``` ### Cache Invalidation The hybrid cache uses Redis pub/sub to invalidate local caches across all instances: ```csharp // When you update a value, all instances are notified await hybridCache.SetAsync("config:app", newConfig); // Other instances automatically invalidate their local copy ``` ### Benefits * **Reduced Latency**: Local cache hits avoid network round-trips * **Reduced Load**: Fewer Redis operations * **Automatic Sync**: Pub/sub keeps caches consistent * **Configurable Size**: Control local cache memory usage ## RedisQueue A reliable distributed queue with visibility timeout and dead letter support. ### Basic Usage ```csharp using Foundatio.Queues; var queue = new RedisQueue(options => { options.ConnectionMultiplexer = redis; options.Name = "work-items"; }); // Enqueue await queue.EnqueueAsync(new WorkItem { Id = 1 }); // Dequeue and process var entry = await queue.DequeueAsync(); try { await ProcessAsync(entry.Value); await entry.CompleteAsync(); } catch { await entry.AbandonAsync(); } ``` ### Configuration Options ```csharp var queue = new RedisQueue(options => { options.ConnectionMultiplexer = redis; options.Name = "work-items"; // Visibility timeout options.WorkItemTimeout = TimeSpan.FromMinutes(5); // Retry settings options.Retries = 3; options.RetryDelay = TimeSpan.FromSeconds(30); // Dead letter settings options.DeadLetterCheckInterval = TimeSpan.FromMinutes(1); options.DeadLetterMaxItems = 100; // Run maintenance (cleanup dead letters) options.RunMaintenanceTasks = true; options.LoggerFactory = loggerFactory; }); ``` ### Queue Features ```csharp // Get queue statistics var stats = await queue.GetQueueStatsAsync(); Console.WriteLine($"Queued: {stats.Queued}"); Console.WriteLine($"Working: {stats.Working}"); Console.WriteLine($"Dead Letter: {stats.Deadlettered}"); // Process continuously await queue.StartWorkingAsync(async (entry, token) => { await ProcessWorkItemAsync(entry.Value); }); ``` ## RedisMessageBus A pub/sub message bus using Redis for cross-process communication. ### Basic Usage ```csharp using Foundatio.Messaging; var messageBus = new RedisMessageBus(options => { options.Subscriber = redis.GetSubscriber(); }); // Subscribe await messageBus.SubscribeAsync(async message => { await HandleOrderCreatedAsync(message); }); // Publish await messageBus.PublishAsync(new OrderCreatedEvent { OrderId = "123" }); ``` ### Configuration Options ```csharp var messageBus = new RedisMessageBus(options => { options.Subscriber = redis.GetSubscriber(); options.Topic = "myapp"; // Channel prefix options.LoggerFactory = loggerFactory; options.Serializer = serializer; }); ``` ### Topic-Based Routing ```csharp // Messages are published to channels based on type // e.g., "myapp:OrderCreatedEvent" // All instances subscribed to this type receive the message await messageBus.SubscribeAsync(HandleOrder); ``` ## RedisFileStorage Store files in Redis (suitable for small files and caching scenarios). ### Basic Usage ```csharp using Foundatio.Storage; var storage = new RedisFileStorage(options => { options.ConnectionMultiplexer = redis; }); // Save file await storage.SaveFileAsync("config/settings.json", settingsJson); // Read file var content = await storage.GetFileContentsAsync("config/settings.json"); ``` ### Configuration Options ```csharp var storage = new RedisFileStorage(options => { options.ConnectionMultiplexer = redis; options.LoggerFactory = loggerFactory; options.Serializer = serializer; }); ``` ::: warning Redis file storage is best for small files or caching scenarios. For large files, consider Azure Blob Storage or S3. ::: ## Distributed Locks with Redis Use `CacheLockProvider` with Redis for distributed locking. ### Basic Usage ```csharp using Foundatio.Lock; var locker = new CacheLockProvider( cache: redisCacheClient, messageBus: redisMessageBus ); await using var lockHandle = await locker.AcquireAsync( resource: "order:123", timeUntilExpires: TimeSpan.FromMinutes(5), cancellationToken: token ); if (lockHandle != null) { // Exclusive access to order:123 await ProcessOrderAsync("123"); } ``` ### Lock Patterns ```csharp // Try to acquire, fail fast var lockHandle = await locker.AcquireAsync("resource"); if (lockHandle == null) { throw new ResourceBusyException(); } // Wait for lock with timeout var lockHandle = await locker.AcquireAsync( resource: "resource", acquireTimeout: TimeSpan.FromSeconds(30) ); // Extend lock duration await lockHandle.RenewAsync(TimeSpan.FromMinutes(5)); ``` ## Complete Redis Setup ### Service Registration ```csharp public static IServiceCollection AddFoundatioRedis( this IServiceCollection services, string connectionString) { // Connection services.AddSingleton(sp => ConnectionMultiplexer.Connect(connectionString)); // Cache (Hybrid for best performance) services.AddSingleton(sp => new RedisHybridCacheClient(options => { options.ConnectionMultiplexer = sp.GetRequiredService(); options.LocalCacheMaxItems = 1000; options.LoggerFactory = sp.GetRequiredService(); })); // Message Bus services.AddSingleton(sp => new RedisMessageBus(options => { options.Subscriber = sp .GetRequiredService() .GetSubscriber(); options.LoggerFactory = sp.GetRequiredService(); })); services.AddSingleton(sp => sp.GetRequiredService()); services.AddSingleton(sp => sp.GetRequiredService()); // Lock Provider services.AddSingleton(sp => new CacheLockProvider( sp.GetRequiredService(), sp.GetRequiredService())); return services; } // Add queue public static IServiceCollection AddRedisQueue( this IServiceCollection services, string name) where T : class { services.AddSingleton>(sp => new RedisQueue(options => { options.ConnectionMultiplexer = sp.GetRequiredService(); options.Name = name; options.LoggerFactory = sp.GetRequiredService(); })); return services; } ``` ### Usage ```csharp var builder = WebApplication.CreateBuilder(args); var redisConnection = builder.Configuration.GetConnectionString("Redis") ?? "localhost:6379"; builder.Services.AddFoundatioRedis(redisConnection); builder.Services.AddRedisQueue("work-items"); builder.Services.AddRedisQueue("emails"); ``` ## Production Considerations ### Connection Resilience ```csharp var options = new ConfigurationOptions { EndPoints = { "redis:6379" }, AbortOnConnectFail = false, // Don't throw on startup ConnectRetry = 5, ReconnectRetryPolicy = new ExponentialRetry(5000) }; ``` ### Health Checks ```csharp builder.Services.AddHealthChecks() .AddRedis(redisConnection, name: "redis"); ``` ### Monitoring ```csharp // Get server info var server = redis.GetServer("redis:6379"); var info = await server.InfoAsync(); // Memory usage var memory = info.FirstOrDefault(g => g.Key == "memory"); ``` ### Cluster Support ```csharp var options = new ConfigurationOptions { EndPoints = { "redis-1:6379", "redis-2:6379", "redis-3:6379" } }; // StackExchange.Redis handles cluster topology automatically var redis = await ConnectionMultiplexer.ConnectAsync(options); ``` ## Best Practices ### 1. Connection Management ```csharp // ✅ Singleton connection services.AddSingleton( ConnectionMultiplexer.Connect("redis:6379")); // ❌ Creating new connections var redis = ConnectionMultiplexer.Connect("redis:6379"); ``` ### 2. Key Naming ```csharp // ✅ Hierarchical, descriptive keys await cache.SetAsync("user:123:profile", profile); await cache.SetAsync("order:456:items", items); // ❌ Flat, ambiguous keys await cache.SetAsync("123", profile); ``` ### 3. Serialization ```csharp // Use efficient serializers for large objects var serializer = new MessagePackSerializer(); var cache = new RedisCacheClient(o => o.Serializer = serializer); ``` ### 4. TTL Strategy ```csharp // Always set expiration for cached data await cache.SetAsync("data", value, TimeSpan.FromHours(1)); // Use sliding expiration for frequently accessed data await cache.SetAsync("session", data, expiresIn: TimeSpan.FromMinutes(30)); ``` ## Next Steps * [Azure Implementation](./azure) - Azure Storage and Service Bus * [AWS Implementation](./aws) - S3 and SQS * [In-Memory Implementation](./in-memory) - Local development --- --- url: /guide/resilience.md --- # Resilience Resilience policies provide a powerful way to handle transient failures and make your applications more robust. Foundatio's resilience system includes retry logic, circuit breakers, timeouts, and exponential backoff. ## The IResiliencePolicy Interface ```csharp public interface IResiliencePolicy { Task ExecuteAsync(Func action, CancellationToken cancellationToken = default); Task ExecuteAsync(Func> action, CancellationToken cancellationToken = default); } ``` ## Basic Usage ### Creating a Policy ```csharp using Foundatio.Resilience; var policy = new ResiliencePolicyBuilder() .WithMaxAttempts(5) .WithExponentialDelay(TimeSpan.FromSeconds(1)) .WithJitter() .Build(); ``` ### Executing with Retry ```csharp await policy.ExecuteAsync(async ct => { await SomeUnreliableOperationAsync(ct); }); ``` ### With Return Values ```csharp var result = await policy.ExecuteAsync(async ct => { return await GetDataFromApiAsync(ct); }); ``` ## ResiliencePolicyBuilder ### Retry Configuration ```csharp var policy = new ResiliencePolicyBuilder() // Maximum number of attempts (default: 3) .WithMaxAttempts(5) // Fixed delay between retries .WithDelay(TimeSpan.FromSeconds(2)) // Or exponential delay (doubles each retry) .WithExponentialDelay(TimeSpan.FromSeconds(1)) // Or linear delay (adds fixed amount each retry) .WithLinearDelay(TimeSpan.FromSeconds(1)) // Maximum delay cap .WithMaxDelay(TimeSpan.FromMinutes(1)) // Add randomness to prevent thundering herd .WithJitter() .Build(); ``` ### Custom Delay Function ```csharp var policy = new ResiliencePolicyBuilder() .WithMaxAttempts(5) .WithDelayFunction(attempt => TimeSpan.FromSeconds(Math.Pow(2, attempt))) .Build(); ``` ### Timeout Configuration ```csharp var policy = new ResiliencePolicyBuilder() .WithMaxAttempts(3) .WithTimeout(TimeSpan.FromSeconds(30)) // Overall timeout .Build(); ``` ### Conditional Retry ```csharp var policy = new ResiliencePolicyBuilder() .WithMaxAttempts(5) .WithExponentialDelay(TimeSpan.FromSeconds(1)) .WithShouldRetry((attempt, exception) => { // Only retry on specific exceptions return exception is HttpRequestException or TimeoutException; }) .Build(); ``` ### Unhandled Exceptions ```csharp var policy = new ResiliencePolicyBuilder() .WithMaxAttempts(5) // These exceptions will be thrown immediately without retry .WithUnhandledException() .WithUnhandledException() .Build(); ``` ## Circuit Breaker Prevent cascading failures by temporarily stopping calls to failing services: ```csharp var policy = new ResiliencePolicyBuilder() .WithMaxAttempts(3) .WithCircuitBreaker(cb => cb // Open circuit after 50% failure rate .WithFailureRatio(0.5) // Need at least 10 calls before evaluating .WithMinimumCalls(10) // Keep circuit open for 1 minute .WithBreakDuration(TimeSpan.FromMinutes(1))) .Build(); ``` ### Circuit Breaker States ```mermaid graph LR A[Closed] -->|Failures exceed threshold| B[Open] B -->|Break duration expires| C[Half-Open] C -->|Test call succeeds| A C -->|Test call fails| B ``` * **Closed**: Normal operation, calls pass through * **Open**: Calls fail immediately without execution * **Half-Open**: Single test call allowed to check recovery ### Checking Circuit State ```csharp var policy = new ResiliencePolicy(); policy.CircuitBreaker = new CircuitBreaker(new CircuitBreakerBuilder() .WithFailureRatio(0.5) .WithMinimumCalls(10) .WithBreakDuration(TimeSpan.FromMinutes(1))); // Check state if (policy.CircuitBreaker.State == CircuitState.Open) { _logger.LogWarning("Circuit is open - skipping operation"); return; } await policy.ExecuteAsync(async ct => { await CallExternalServiceAsync(ct); }); ``` ## ResiliencePolicy Properties ```csharp var policy = new ResiliencePolicy { // Maximum retry attempts MaxAttempts = 5, // Fixed delay between retries Delay = TimeSpan.FromSeconds(2), // Custom delay function GetDelay = ResiliencePolicy.ExponentialDelay(TimeSpan.FromSeconds(1)), // Maximum delay cap MaxDelay = TimeSpan.FromMinutes(1), // Add jitter to delays UseJitter = true, // Overall timeout Timeout = TimeSpan.FromMinutes(5), // Circuit breaker CircuitBreaker = new CircuitBreaker(...), // Exceptions that should not be retried UnhandledExceptions = { typeof(OperationCanceledException) }, // Custom retry logic ShouldRetry = (attempt, ex) => ex is TransientException, // Logger for retry events Logger = logger }; ``` ## Static Delay Functions Foundatio provides built-in delay functions: ```csharp // Exponential: 1s, 2s, 4s, 8s, 16s... ResiliencePolicy.ExponentialDelay(TimeSpan.FromSeconds(1)) // Linear: 1s, 2s, 3s, 4s, 5s... ResiliencePolicy.LinearDelay(TimeSpan.FromSeconds(1)) // Constant: 2s, 2s, 2s, 2s... ResiliencePolicy.ConstantDelay(TimeSpan.FromSeconds(2)) ``` ## IResiliencePolicyProvider Manage multiple named policies: ```csharp using Foundatio.Resilience; var provider = new ResiliencePolicyProviderBuilder() // Default policy for unspecified operations .WithDefaultPolicy(builder => builder .WithMaxAttempts(3) .WithExponentialDelay(TimeSpan.FromSeconds(1))) // Named policy for external APIs .WithPolicy("external-api", builder => builder .WithMaxAttempts(5) .WithCircuitBreaker() .WithTimeout(TimeSpan.FromSeconds(30))) // Named policy for database operations .WithPolicy("database", builder => builder .WithMaxAttempts(3) .WithLinearDelay(TimeSpan.FromMilliseconds(100)) .WithUnhandledException()) .Build(); // Get and use policies var apiPolicy = provider.GetPolicy("external-api"); await apiPolicy.ExecuteAsync(async ct => { await CallExternalApiAsync(ct); }); var dbPolicy = provider.GetPolicy("database"); await dbPolicy.ExecuteAsync(async ct => { await SaveToDbAsync(ct); }); ``` ## Type-Based Policies Get policies based on service type: ```csharp var provider = new ResiliencePolicyProviderBuilder() .WithPolicy(builder => builder .WithMaxAttempts(5) .WithCircuitBreaker()) .WithPolicy(builder => builder .WithMaxAttempts(3) .WithLinearDelay()) .Build(); var policy = provider.GetPolicy(); ``` ## Common Patterns ### HTTP Client with Resilience ```csharp public class ResilientHttpClient { private readonly HttpClient _client; private readonly IResiliencePolicy _policy; public ResilientHttpClient(HttpClient client) { _client = client; _policy = new ResiliencePolicyBuilder() .WithMaxAttempts(3) .WithExponentialDelay(TimeSpan.FromSeconds(1)) .WithCircuitBreaker(cb => cb .WithFailureRatio(0.5) .WithMinimumCalls(10) .WithBreakDuration(TimeSpan.FromMinutes(1))) .WithTimeout(TimeSpan.FromSeconds(30)) .WithShouldRetry((_, ex) => ex is HttpRequestException or TaskCanceledException) .Build(); } public async Task GetAsync(string url) { return await _policy.ExecuteAsync(async ct => { var response = await _client.GetAsync(url, ct); response.EnsureSuccessStatusCode(); return await response.Content.ReadFromJsonAsync(ct); }); } } ``` ### Database Operations ```csharp public class ResilientRepository { private readonly IResiliencePolicy _policy; public ResilientRepository() { _policy = new ResiliencePolicyBuilder() .WithMaxAttempts(3) .WithLinearDelay(TimeSpan.FromMilliseconds(100)) .WithShouldRetry((_, ex) => IsTransientDbException(ex)) .Build(); } public async Task GetUserAsync(int id) { return await _policy.ExecuteAsync(async ct => { return await _context.Users.FindAsync(id, ct); }); } private bool IsTransientDbException(Exception ex) { return ex is DbUpdateException or TimeoutException; } } ``` ### Graceful Degradation ```csharp public class CatalogService { private readonly IResiliencePolicy _policy; private readonly ICacheClient _cache; public async Task GetProductAsync(int id) { try { return await _policy.ExecuteAsync(async ct => { return await _api.GetProductAsync(id, ct); }); } catch (BrokenCircuitException) { // Fall back to cached data when circuit is open var cached = await _cache.GetAsync($"product:{id}"); if (cached.HasValue) { _logger.LogWarning("Returning cached product {Id}", id); return cached.Value; } throw; } } } ``` ### Retry with Logging ```csharp var policy = new ResiliencePolicy { MaxAttempts = 5, GetDelay = ResiliencePolicy.ExponentialDelay(TimeSpan.FromSeconds(1)), Logger = loggerFactory.CreateLogger("Resilience") }; // Logs will include attempt number, delay, and exception details await policy.ExecuteAsync(async ct => { await UnreliableOperationAsync(ct); }); ``` ## Integration with Foundatio ### Cache with Resilience ```csharp var resilientCache = new ResilientCacheClient( new RedisCacheClient(...), new ResiliencePolicyBuilder() .WithMaxAttempts(3) .WithExponentialDelay(TimeSpan.FromMilliseconds(100)) .Build() ); ``` ### Queue with Resilience ```csharp public class ResilientQueueProcessor { private readonly IQueue _queue; private readonly IResiliencePolicy _policy; public async Task ProcessAsync(WorkItem item) { await _policy.ExecuteAsync(async ct => { await DoProcessingAsync(item, ct); }); } } ``` ## Dependency Injection ### Register Policy Provider ```csharp services.AddSingleton(sp => { var logger = sp.GetRequiredService().CreateLogger("Resilience"); return new ResiliencePolicyProviderBuilder() .WithDefaultPolicy(b => b .WithMaxAttempts(3) .WithExponentialDelay(TimeSpan.FromSeconds(1)) .WithLogger(logger)) .WithPolicy("http", b => b .WithMaxAttempts(5) .WithCircuitBreaker() .WithTimeout(TimeSpan.FromSeconds(30))) .Build(); }); ``` ### Use in Services ```csharp public class MyService { private readonly IResiliencePolicy _policy; public MyService(IResiliencePolicyProvider policyProvider) { _policy = policyProvider.GetPolicy("http"); } } ``` ## Best Practices ### 1. Use Appropriate Timeouts ```csharp // Match timeout to operation type .WithTimeout(TimeSpan.FromSeconds(5)) // Fast operations .WithTimeout(TimeSpan.FromSeconds(30)) // API calls .WithTimeout(TimeSpan.FromMinutes(5)) // Long operations ``` ### 2. Configure Circuit Breaker Thresholds ```csharp .WithCircuitBreaker(cb => cb // High traffic: needs more samples .WithMinimumCalls(100) .WithFailureRatio(0.5) // Low traffic: fewer samples .WithMinimumCalls(10) .WithFailureRatio(0.3) ) ``` ### 3. Use Jitter to Prevent Thundering Herd ```csharp .WithExponentialDelay(TimeSpan.FromSeconds(1)) .WithJitter() // Adds randomness to prevent synchronized retries ``` ### 4. Handle Specific Exceptions ```csharp .WithShouldRetry((attempt, ex) => { // Only retry transient failures return ex is HttpRequestException or TimeoutException or SocketException; }) .WithUnhandledException() .WithUnhandledException() ``` ### 5. Log Retry Attempts ```csharp var policy = new ResiliencePolicy { Logger = loggerFactory.CreateLogger("Resilience"), MaxAttempts = 5 }; // Automatically logs each retry attempt ``` ## Next Steps * [Caching](./caching) - Combine with cache fallbacks * [Queues](./queues) - Resilient queue processing * [Jobs](./jobs) - Retry job execution --- --- url: /guide/what-is-foundatio.md --- # What is Foundatio? Foundatio is a modular .NET library providing pluggable building blocks for distributed applications, including: * **Caching** - Fast data access with multiple backend implementations * **Queues** - FIFO message delivery for background processing * **Locks** - Distributed locking for resource coordination * **Messaging** - Pub/sub patterns for event-driven architectures * **Jobs** - Long-running process management * **File Storage** - Abstracted file operations * **Resilience** - Retry policies and circuit breakers ## Design Philosophy Foundatio was built with several key principles in mind: ### Abstract Interfaces All core functionality is exposed through clean interfaces (`ICacheClient`, `IQueue`, `ILockProvider`, `IMessageBus`, `IFileStorage`). This allows you to: * **Swap implementations** without changing application code * **Test easily** using in-memory implementations * **Scale gradually** by switching to distributed implementations when needed ### Dependency Injection First Every component is designed to work seamlessly with Microsoft.Extensions.DependencyInjection: ```csharp services.AddSingleton(sp => new InMemoryCacheClient()); services.AddSingleton(sp => new InMemoryMessageBus()); services.AddSingleton(sp => new CacheLockProvider( sp.GetRequiredService(), sp.GetRequiredService() )); ``` ### Development-Production Parity In-memory implementations for all abstractions mean: * **No external dependencies** during development * **Fast unit tests** without infrastructure setup * **Same code paths** in development and production ### Extensibility Each abstraction can be extended with custom implementations: ```csharp public class MyCustomCacheClient : ICacheClient { // Your custom implementation } ``` ## Core Abstractions ### Caching Store and retrieve data with expiration support: ```csharp ICacheClient cache = new InMemoryCacheClient(); await cache.SetAsync("user:123", user, TimeSpan.FromMinutes(30)); var cached = await cache.GetAsync("user:123"); ``` [Learn more about Caching →](./caching) ### Queues Reliable message delivery with at-least-once semantics: ```csharp IQueue queue = new InMemoryQueue(); await queue.EnqueueAsync(new WorkItem { Id = 1 }); var entry = await queue.DequeueAsync(); // Process and complete await entry.CompleteAsync(); ``` [Learn more about Queues →](./queues) ### Locks Distributed locking for coordinating access: ```csharp ILockProvider locker = new CacheLockProvider(cache, messageBus); await using var @lock = await locker.AcquireAsync("my-resource"); if (@lock != null) { // Exclusive access to resource } ``` [Learn more about Locks →](./locks) ### Messaging Publish/subscribe messaging: ```csharp IMessageBus bus = new InMemoryMessageBus(); await bus.SubscribeAsync(msg => ProcessOrder(msg)); await bus.PublishAsync(new OrderCreated { OrderId = 123 }); ``` [Learn more about Messaging →](./messaging) ### File Storage Abstracted file operations: ```csharp IFileStorage storage = new FolderFileStorage("/data"); await storage.SaveFileAsync("reports/2024/report.pdf", fileStream); var file = await storage.GetFileStreamAsync("reports/2024/report.pdf"); ``` [Learn more about Storage →](./storage) ### Jobs Background job processing: ```csharp public class MyJob : JobBase { protected override Task RunInternalAsync(JobContext context) { // Do work return Task.FromResult(JobResult.Success); } } ``` [Learn more about Jobs →](./jobs) ### Resilience Retry policies with circuit breakers: ```csharp var policy = new ResiliencePolicyBuilder() .WithMaxAttempts(5) .WithExponentialDelay(TimeSpan.FromSeconds(1)) .WithCircuitBreaker() .Build(); await policy.ExecuteAsync(async ct => { await SomeUnreliableOperationAsync(ct); }); ``` [Learn more about Resilience →](./resilience) ## When to Use Foundatio ### ✅ Great For * **Microservices** with distributed caching, queuing, and messaging needs * **Background processing** with reliable job execution * **Event-driven architectures** using pub/sub patterns * **Cloud applications** needing portable abstractions * **Development teams** wanting consistent patterns across projects * **Testing scenarios** requiring isolated, fast-running tests ### ⚠️ Consider Alternatives For * **Simple applications** with no distributed requirements * **Projects already invested** in specific vendor SDKs * **Extremely high-throughput scenarios** where direct SDK access is needed ## Next Steps Ready to get started? Here's what to explore next: * [Getting Started](./getting-started) - Install and configure Foundatio * [Caching](./caching) - Deep dive into caching * [Why Choose Foundatio?](./why-foundatio) - Detailed comparison and benefits --- --- url: /guide/why-foundatio.md --- # Why Choose Foundatio? Foundatio was born from real-world experience building large-scale cloud applications. Here's why it stands out from other approaches. ## The Problem When building distributed applications, you typically face these challenges: 1. **Vendor Lock-in**: Direct use of Redis, Azure, or AWS SDKs couples your code to specific providers 2. **Testing Complexity**: External dependencies make testing slow and unreliable 3. **Development Setup**: Developers need to run Redis, Azure Storage Emulator, etc. locally 4. **Inconsistent Patterns**: Different team members use different approaches for the same problems 5. **Reinventing the Wheel**: Every project implements caching, queuing, locking from scratch ## The Foundatio Solution ### 🔌 Pluggable Abstractions Write your code against interfaces, not implementations: ```csharp // Your service doesn't care about the implementation public class OrderProcessor { private readonly ICacheClient _cache; private readonly IQueue _queue; public OrderProcessor(ICacheClient cache, IQueue queue) { _cache = cache; _queue = queue; } } ``` Change from in-memory to Redis with one line in your DI configuration: ```csharp // Development services.AddSingleton(); // Production services.AddSingleton(sp => new RedisCacheClient(o => o.ConnectionMultiplexer = redis)); ``` ### 🧪 Superior Testing Experience No mocking frameworks needed - use real implementations: ```csharp [Fact] public async Task Should_Process_Order_With_Caching() { // Arrange - use in-memory implementations var cache = new InMemoryCacheClient(); var queue = new InMemoryQueue(); var processor = new OrderProcessor(cache, queue); // Act await processor.ProcessAsync(new Order { Id = 1 }); // Assert var cached = await cache.GetAsync("order:1"); Assert.NotNull(cached); } ``` Benefits: * **Fast**: No network calls * **Isolated**: No shared state between tests * **Reliable**: No external service failures * **Complete**: Test the exact code path used in production ### 🚀 Zero-Config Development Start coding immediately without external dependencies: ```csharp // Works out of the box - no Redis, no Azure, no AWS var cache = new InMemoryCacheClient(); var queue = new InMemoryQueue(); var messageBus = new InMemoryMessageBus(); var storage = new InMemoryFileStorage(); ``` Compare to other approaches: * **Direct Redis**: Requires running Redis server * **Direct Azure**: Requires Azure subscription or emulator * **Direct AWS**: Requires AWS account or LocalStack ### 📊 Comparison with Alternatives #### vs. Direct SDK Usage | Aspect | Direct SDK | Foundatio | |--------|------------|-----------| | Testing | Mock everything | Use in-memory implementations | | Switching providers | Rewrite code | Change DI registration | | Local development | Run services | Zero dependencies | | Learning curve | Learn each SDK | Learn one API | #### vs. Building Your Own | Aspect | DIY | Foundatio | |--------|-----|-----------| | Time to implement | Weeks/months | Minutes | | Battle tested | No | Yes (years of production use) | | Edge cases handled | Maybe | Comprehensive | | Maintenance burden | On you | On community | #### vs. Other Libraries Foundatio is unique in providing: 1. **Complete abstraction set**: Caching, Queues, Locks, Messaging, Storage, Jobs, Resilience 2. **Consistent API design**: Same patterns across all abstractions 3. **In-memory implementations**: For all abstractions, not just some 4. **Active maintenance**: Regular updates and community support ## Real-World Usage ### Exceptionless [Exceptionless](https://github.com/exceptionless/Exceptionless), a large-scale error tracking application, uses Foundatio extensively: * **Caching**: User sessions, resolved geo-locations with `MaxItems` limit * **Queues**: Event processing pipeline * **Jobs**: Background processing for reports, cleanup, notifications * **Storage**: Error stack traces, attachments * **Messaging**: Real-time notifications ### Enterprise Applications Teams choose Foundatio for: * **Consistency**: Same patterns across all services * **Onboarding**: New developers learn one API * **Migration**: Easy to switch providers without code changes * **Testing**: Comprehensive test coverage without infrastructure ## Feature Highlights ### Hybrid Caching Combine local and distributed caching for maximum performance: ```csharp var hybridCache = new HybridCacheClient( distributedCache: new RedisCacheClient(...), messageBus: new RedisMessageBus(...) ); ``` * Local cache for fastest access * Distributed cache for consistency * Message bus for cache invalidation ### Scoped Caching Easily namespace your cache keys: ```csharp var scopedCache = new ScopedCacheClient(cache, "tenant:123"); await scopedCache.SetAsync("user", user); // Key: "tenant:123:user" await scopedCache.RemoveByPrefixAsync(""); // Clears all tenant:123 keys ``` ### Throttling Locks Rate limit operations across all instances: ```csharp var throttledLocker = new ThrottlingLockProvider( cache, maxHits: 10, period: TimeSpan.FromMinutes(1) ); // Only allows 10 operations per minute across all instances if (await throttledLocker.AcquireAsync("api-call")) { await CallExternalApiAsync(); } ``` ### Queue Behaviors Extend queue functionality with behaviors: ```csharp queue.AttachBehavior(new MetricsQueueBehavior(metrics)); queue.AttachBehavior(new RetryQueueBehavior(maxRetries: 3)); ``` ### Resilience Policies Built-in retry and circuit breaker: ```csharp var policy = new ResiliencePolicyBuilder() .WithMaxAttempts(5) .WithExponentialDelay(TimeSpan.FromSeconds(1)) .WithMaxDelay(TimeSpan.FromMinutes(1)) .WithJitter() .WithCircuitBreaker(cb => cb .WithFailureRatio(0.5) .WithBreakDuration(TimeSpan.FromMinutes(1))) .Build(); ``` ## Getting Started Ready to try Foundatio? 1. [Installation & Setup](./getting-started) - Get running in minutes 2. [Caching Guide](./caching) - Deep dive into caching 3. [Sample Application](https://github.com/FoundatioFx/Foundatio.Samples) - See complete examples The combination of consistent abstractions, excellent testability, and production-ready implementations makes Foundatio an excellent choice for modern .NET applications.