flashQ in Action: Live Demo of All Features

Let's walk through flashQ's features with practical examples. We'll simulate real scenarios showing exactly how each feature works, from basic job processing to advanced patterns.

🚀 1. Basic Job Processing

Let's start with the fundamentals: pushing a job and processing it.

// producer.ts
const client = new FlashQ();
await client.connect();

const job = await client.push('emails', {
  to: 'user@example.com',
  subject: 'Welcome!',
  body: 'Thanks for signing up.'
});

console.log(`Job created: #${job.id}`);

⚡ 2. Priority Queues

High-priority jobs get processed first. Let's see how priorities work:

// Critical job - processed first
await client.push('tasks', { type: 'urgent' }, { priority: 100 });

// Normal job
await client.push('tasks', { type: 'regular' }, { priority: 10 });

// Background job - processed last
await client.push('tasks', { type: 'cleanup' }, { priority: 1 });

⏱️ 3. Delayed Jobs

Schedule jobs to run in the future:

// Send reminder in 5 seconds
await client.push('reminders', {
  message: "Don't forget your meeting!"
}, {
  delay: 5000  // 5000ms = 5 seconds
});

// Schedule for specific time
const tomorrow = new Date();
tomorrow.setDate(tomorrow.getDate() + 1);

await client.push('reports', { type: 'daily' }, {
  delay: tomorrow.getTime() - Date.now()
});

🔄 4. Automatic Retries with Exponential Backoff

When jobs fail, flashQ automatically retries them with exponential backoff:

await client.push('api-calls', { url: 'https://api.example.com' }, {
  max_attempts: 3,   // Try up to 3 times
  backoff: 1000      // Start with 1s, then 2s, then 4s...
});

💀 5. Dead Letter Queue (DLQ)

Jobs that fail all attempts go to the DLQ for inspection:

// Get failed jobs
const failedJobs = await client.getDlq('payments', 10);

for (const job of failedJobs) {
  console.log(`Job #${job.id} failed: ${job.failedReason}`);

  // Retry specific job
  await client.retryDlq('payments', job.id);
}

// Retry all DLQ jobs
await client.retryDlq('payments');

🚦 6. Rate Limiting

Control how fast jobs are processed to respect API limits:

// Limit to 10 jobs per second
await client.setRateLimit('openai', {
  max: 10,
  duration: 1000  // 1 second
});

// Limit to 100 jobs per minute
await client.setRateLimit('emails', {
  max: 100,
  duration: 60000  // 60 seconds
});

📊 7. Progress Tracking

Track progress for long-running jobs:

// Worker updates progress
new Worker('documents', async (job) => {
  const docs = job.data.documents;
  const batchSize = 100;

  for (let i = 0; i < docs.length; i += batchSize) {
    await processBatch(docs.slice(i, i + batchSize));

    // Update progress
    const percent = Math.round((i + batchSize) / docs.length * 100);
    await job.updateProgress(percent, `Processing batch ${i/batchSize + 1}`);
  }

  return { processed: docs.length };
});

// Monitor progress from another process
const progress = await client.getProgress(jobId);
console.log(`${progress.percent}% - ${progress.message}`);

⏰ 8. Cron Jobs

Schedule recurring jobs with cron expressions:

// Daily report at 9 AM
await client.addCron('daily-report', {
  queue: 'reports',
  schedule: '0 9 * * *',
  data: { type: 'daily' }
});

// Every 5 minutes
await client.addCron('cleanup', {
  queue: 'maintenance',
  schedule: '*/5 * * * *',
  data: { action: 'cleanup' }
});

// List all cron jobs
const crons = await client.listCrons();
crons.forEach(c => console.log(`${c.name}: ${c.schedule}`));

🔗 9. Job Dependencies (Workflows)

Create complex workflows where jobs depend on other jobs:

// Create workflow with dependencies
const extract = await client.push('pipeline', {
  step: 'extract',
  file: 'document.pdf'
});

const embed = await client.push('pipeline', {
  step: 'embed'
}, {
  depends_on: [extract.id]  // Wait for extract
});

const store = await client.push('pipeline', {
  step: 'store'
}, {
  depends_on: [embed.id]  // Wait for embed
});

// Wait for entire workflow
const result = await client.finished(store.id);
console.log('Workflow complete!', result);

📈 10. Real-Time Monitoring

Monitor your queues with live metrics:

// Get queue statistics
const stats = await client.stats();
console.log(`Total queues: ${stats.queues}`);
console.log(`Jobs waiting: ${stats.waiting}`);
console.log(`Jobs active: ${stats.active}`);

// Get detailed metrics
const metrics = await client.metrics();
console.log(`Throughput: ${metrics.throughput}/min`);
console.log(`Avg latency: ${metrics.avgLatency}ms`);

// Get job counts by state
const counts = await client.getJobCounts('emails');
console.log(counts);
// { waiting: 23, active: 5, completed: 12847, failed: 2 }

🎯 Summary

flashQ provides a complete toolkit for background job processing:

Feature	Use Case
Priority Queues	Process urgent jobs first
Delayed Jobs	Schedule for future execution
Retries + Backoff	Handle transient failures
Dead Letter Queue	Debug and retry failed jobs
Rate Limiting	Respect API limits
Progress Tracking	Monitor long-running jobs
Cron Jobs	Recurring scheduled tasks
Dependencies	Complex workflows
Real-Time Metrics	Monitor queue health