Background Jobs and Scheduled Tasks

The Stratpoint Timesheet Application uses Laravel's scheduled commands and a limited queue system for background processing. Most heavy operations are handled through scheduled Artisan commands rather than queue jobs. This section details the actual background processing implementation.

Queue Architecture Overview

Queue System Design

graph TB
    A[Web Request] --> B[Controller]
    B --> C{Immediate Response?}
    C -->|Yes| D[Synchronous Processing]
    C -->|No| E[Queue Job]

    E --> F[Job Queue]
    F --> G[Queue Worker]
    G --> H[Job Processing]
    H --> I{Success?}
    I -->|Yes| J[Job Completed]
    I -->|No| K[Retry Logic]
    K --> L{Max Retries?}
    L -->|No| F
    L -->|Yes| M[Failed Job]

    N[Queue Types] --> O[High Priority]
    N --> P[Default]
    N --> Q[Low Priority]
    N --> R[Email]
    N --> S[Reports]

Queue Configuration

Queue Connections:

// config/queue.php
'connections' => [
    'redis' => [
        'driver' => 'redis',
        'connection' => 'default',
        'queue' => env('REDIS_QUEUE', 'default'),
        'retry_after' => 90,
        'block_for' => null,
    ],

    'database' => [
        'driver' => 'database',
        'table' => 'jobs',
        'queue' => 'default',
        'retry_after' => 90,
    ],
]

Queue Priorities: - high: Critical system operations (1-5 seconds) - default: Standard business operations (5-30 seconds) - low: Background maintenance tasks (30+ seconds) - emails: Email notifications and communications - reports: Report generation and data processing

Actual Background Processing Implementation

Queue Jobs (Limited Use)

The application has a limited number of actual queue jobs:

Existing Job Classes:

// Real job classes in app/Jobs/
- BatchProcessAwardedLeaves
- CreateProjectMarginDashboardSnapshot
- CreateRevenueSnapshot
- CreateTimelogDeficiencySnapshot
- DeactivateResignedEmployeesToday
- DeletePastExportedFiles
- EmailOffsetSummaryForTheMonth
- EmailDetailedTimelogsAndCtoReports
- EmailEndingAssignments
- EmailExcessHours
- NotifyProjectsNearDeadline
- ScanFile

Example Real Job Implementation:

<?php

namespace App\Jobs;

use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;

class EmailOffsetSummaryForTheMonth implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    protected $data = [];
    public $uniqueFor = 3600;

    public function __construct($inputDate = null)
    {
        $this->data['inputDate'] = $inputDate;
    }

    public function handle()
    {
        $params = ['date' => $this->data['inputDate']];
        $controller = app()->make('App\Http\Controllers\TimelogController');
        app()->call([$controller, 'emailOffsetSummaryForTheMonth'], $params);
    }
}

Artisan Commands (Primary Background Processing)

Most background processing is handled through scheduled Artisan commands:

Available Commands:

// app/Console/Commands/
- NotifyFinanceWhenUserChanged
- NotifyProjectsNearDeadline
- EmailAboutGenericTaskTimelogs
- ProcessLatestOnsiteTimes
- CreateRevenueSnapshot
- EmailHrSoonToBeRegularized
- ProcessAwardedLeaveDiscrepancy
- DeletePastExportedFiles
- DeactivateResignedEmployeesToday
- EmailOffsetSummaryForTheMonth
- EmailProjectUpdates
- CreateProjectMarginDashboardSnapshot
- BatchProcessAwardedLeaves
- CreateTimelogDeficiencySnapshot
- EmailDetailedTimelogsAndCtoReports
- EmailExcessHours
- EmailEndingAssignments
- EmailDeficiencyLevel1/2/3

Email Processing

Email notifications are handled through scheduled Artisan commands rather than queue jobs:

Email Commands: - EmailOffsetSummaryForTheMonth: Weekly CTO offset summaries - EmailDetailedTimelogsAndCtoReports: Monthly detailed reports - EmailExcessHours: Monthly excess hours notifications - EmailEndingAssignments: Daily ending assignment alerts - EmailHrSoonToBeRegularized: Daily regularization reminders - EmailAboutGenericTaskTimelogs: Daily generic task alerts - EmailProjectUpdates: Daily project status updates

Email Implementation Pattern: Most email commands call controller methods directly:

public function handle()
{
    $controller = app()->make('App\Http\Controllers\TimelogController');
    app()->call([$controller, 'emailOffsetSummaryForTheMonth'], $params);
}

Data Snapshot and Report Generation

Snapshot creation is handled through scheduled commands:

Snapshot Commands: - CreateRevenueSnapshot: Weekly revenue snapshots - CreateProjectMarginDashboardSnapshot: Weekly project margin snapshots - CreateTimelogDeficiencySnapshot: Weekly timelog deficiency snapshots - CreateDealProjectRevenueSnapshot: Weekly deal project revenue snapshots

Implementation Pattern: These commands typically create database snapshots for historical reporting:

// Example from actual codebase pattern
class CreateRevenueSnapshot extends Command
{
    protected $signature = 'timesheet:createRevenueSnapshot {includeBreakdown?}';
    protected $description = 'Create revenue snapshot for historical data';

    public function handle()
    {
        // Snapshot creation logic
    }
}

Maintenance and Cleanup Tasks

Maintenance Commands: - DeletePastExportedFiles: Daily cleanup of old exported files - DeactivateResignedEmployeesToday: Daily deactivation of resigned employees - ProcessExpiredCto: Daily processing of expired CTO - BatchChangeEmploymentType: Daily employment type updates - BatchProcessAwardedLeaves: Daily leave processing - FixLeavesEarnedMoreThanAwarded: Daily leave balance corrections

Data Processing Commands: - ProcessLatestOnsiteTimes: Process onsite time data - ExtractAssignmentsWithCost: Extract assignment cost data - UpdateIntercoPositionOfUsers: Update intercompany positions

System Maintenance

Built-in Laravel Maintenance:

// From actual Kernel.php schedule
$schedule->command('log:clear')->daily();

Note: No dedicated cleanup jobs exist for tokens or sessions - the application relies on Laravel's built-in mechanisms and database-level cleanup where needed.

Job Scheduling and Automation

Scheduled Jobs

Actual Scheduler Configuration:

// From app/Console/Kernel.php
protected function schedule(Schedule $schedule)
{
    $schedule->command('log:clear')->daily();

    // Weekly snapshots
    $schedule->command('timesheet:createRevenueSnapshot 0')->weekly()->mondays()->at('14:00')->onOneServer();
    $schedule->command('timesheet:createRevenueSnapshot 1')->weekly()->mondays()->at('14:10')->onOneServer();
    $schedule->command('timesheet:createDealProjectRevenueSnapshot')->weekly()->fridays()->at('23:00')->onOneServer();
    $schedule->command('timesheet:createProjectMarginDashboardSnapshot')->weekly()->sundays()->at('23:00')->onOneServer();
    $schedule->command('timesheet:createTimelogDeficiencySnapshot')->weekly()->saturdays()->at('23:30')->onOneServer();

    // Daily maintenance
    $schedule->command('timesheet:deletePastExportedFiles')->dailyAt('01:00')->onOneServer();
    $schedule->command('timesheet:deactivateResignedEmployeesToday')->daily()->onOneServer();
    $schedule->command('timesheet:processExpiredCto')->daily()->onOneServer();
    $schedule->command('timesheet:batchChangeEmploymentType')->dailyAt('01:00')->onOneServer();
    $schedule->command('timesheet:batchProcessAwardedLeaves')->dailyAt('02:00')->onOneServer();

    // Production-only email notifications
    if(strtolower(config('app.env')) == 'prod'){
        $schedule->command('timesheet:notifyProjectsNearDeadline')->dailyAt('08:00')->onOneServer();
        $schedule->command('timesheet:emailAboutGenericTaskTimelogs')->dailyAt('09:00')->onOneServer();
        $schedule->command('timesheet:emailHrSoonToBeRegularized')->dailyAt('10:00')->onOneServer();
        $schedule->command('timesheet:emailOffsetSummaryForTheMonth')->weekly()->mondays()->at('20:00')->onOneServer();
        $schedule->command('timesheet:emailProjectUpdates')->dailyAt('01:30')->onOneServer();
        $schedule->command('timesheet:emailDetailedTimelogsAndCtoReports')->monthlyOn(1, '04:00')->onOneServer();
        $schedule->command('timesheet:emailExcessHours')->monthlyOn(1, '06:00')->onOneServer();
        $schedule->command('timesheet:emailEndingAssignments')->dailyAt('08:30')->onOneServer();
        $schedule->command('timesheet:emailOnsiteTimeAttendance')->weekly()->fridays()->at('17:00')->onOneServer();
    }
}

Job Monitoring and Management

Queue Worker Configuration:

# Supervisor configuration for queue workers
[program:timesheet-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/timesheet/artisan queue:work redis --sleep=3 --tries=3 --max-time=3600
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=www-data
numprocs=4
redirect_stderr=true
stdout_logfile=/var/log/timesheet-worker.log
stopwaitsecs=3600

Queue Monitoring Commands:

# Monitor queue status
php artisan queue:monitor redis:default,redis:high,redis:emails

# Process specific queue
php artisan queue:work --queue=high,default,low

# List failed jobs
php artisan queue:failed

# Retry failed jobs
php artisan queue:retry all

# Clear all jobs
php artisan queue:clear

# Restart queue workers
php artisan queue:restart

Error Handling and Recovery

Failed Job Handling:

// config/queue.php
'failed' => [
    'driver' => 'database',
    'database' => env('DB_CONNECTION', 'mysql'),
    'table' => 'failed_jobs',
],

Job Retry Logic:

public function retryUntil()
{
    return now()->addMinutes(5);
}

public function backoff()
{
    return [1, 5, 10]; // Retry after 1, 5, and 10 seconds
}

This comprehensive background job system ensures reliable, scalable processing of asynchronous tasks while maintaining system performance and user experience in the Stratpoint Timesheet Application.