Cronjobs
Overview
Scheduled background tasks (cron jobs) are essential for automating recurring operations in your application. This guide covers implementation approaches for containerized applications deployed on the Sherpa.sh platform.
How Cron Jobs Work on Sherpa.sh
Platform Architecture
On Sherpa.sh, your applications run as containerized workloads managed by Kubernetes clusters. This architecture provides the perfect foundation for running reliable background cron jobs alongside your web application.
Key Platform Benefits:
Automatic Scaling: Kubernetes automatically scales your containers based on resource usage
High Availability: Multiple redundant control planes ensure your cron jobs continue running
Container Isolation: Jobs run in identical environments across all instances
Global Distribution: Jobs can run across multiple regions for reliability
Deployment Model
Your application is deployed as a Docker container with the following characteristics:
Single Container: Both your web server and cron jobs run in the same container instance
Persistent Process: The container's entrypoint keeps both services running simultaneously
Shared Resources: Web requests and background jobs share the same application context
Auto-restart: Kubernetes automatically restarts containers if they fail
Request vs Background Job Execution
Understanding the execution model is crucial for proper cron job implementation:
HTTP Requests
60 seconds
API endpoints, page rendering
Background Jobs
Indefinite
Data processing, cleanup, reports
This means your cron jobs can run for hours while HTTP requests are capped at 60 seconds.
Quick Setup Guide
Step 1: Configure Your Container Entrypoint
Structure your application to handle both web traffic and scheduled tasks:
// server.js - Main application entry point
const express = require('express');
const cron = require('node-cron');
const app = express();
// Web server setup
app.use(express.json());
app.get('/health', (req, res) => {
res.json({ status: 'healthy', timestamp: new Date().toISOString() });
});
// Initialize cron jobs
function initializeCronJobs() {
// Daily cleanup at 2 AM UTC
cron.schedule('0 2 * * *', async () => {
console.log('Starting daily cleanup...');
await performDailyCleanup();
});
// Hourly health check
cron.schedule('0 * * * *', async () => {
await performHealthCheck();
});
console.log('Cron jobs initialized');
}
// Start both web server and cron jobs
const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
console.log(`Server running on port ${PORT}`);
initializeCronJobs
Step 2: Update Your Dockerfile
Ensure your container is configured for long-running processes:
FROM node:18-alpine
WORKDIR /app
# Install dependencies
COPY package*.json ./
RUN npm install
# Copy application code
COPY . .
# Expose port for web traffic
EXPOSE 3000
# Start both web server and cron jobs
CMD ["node", "server.js"]
Step 3: Deploy to Sherpa.sh
Deploy your application normally - the platform will automatically:
Build your Docker container
Deploy to Kubernetes clusters
Start your web server and cron jobs
Scale based on traffic and resource usage
Implementation Strategies
Prerequisites
Active Sherpa.sh account
Supported runtime: Node.js, Python, Go, or other containerized applications
Git repository connected to Sherpa.sh
Basic understanding of cron syntax
Platform-Specific Considerations
Resource Management
Your cron jobs share resources with your web application. Monitor memory and CPU usage to ensure optimal performance:
// Monitor resource usage in cron jobs
cron.schedule('0 1 * * *', async () => {
const startMemory = process.memoryUsage();
console.log(`Job started - Memory: ${Math.round(startMemory.heapUsed / 1024 / 1024)}MB`);
await performDataProcessing();
const endMemory = process.memoryUsage();
console.log(`Job completed - Memory: ${Math.round(endMemory.heapUsed / 1024 / 1024)}MB`);
});
Environment Variables
Access your application's environment variables within cron jobs:
javascriptcron.schedule('0 */6 * * *', async () => {
const apiKey = process.env.EXTERNAL_API_KEY;
const environment = process.env.NODE_ENV;
if (environment === 'production') {
await syncProductionData(apiKey);
}
});
Logging Integration
Leverage Sherpa.sh's logging infrastructure for cron job monitoring:
javascriptcron.schedule('0 3 * * *', async () => {
try {
console.log('[CRON] Starting backup process');
await performBackup();
console.log('[CRON] Backup completed successfully');
} catch (error) {
console.error('[CRON] Backup failed:', error.message);
// Logs are automatically captured by Sherpa.sh monitoring
}
});
Basic Cron Implementation
Any standard cron library works well within this architecture:
Node.js Example
const cron = require('node-cron');
// Daily backup at 3 AM
cron.schedule('0 3 * * *', async () => {
await performBackup();
});
// Hourly health check
cron.schedule('0 * * * *', async () => {
await healthCheck();
});
Python Example
Python Code for Scheduling Weekly Reports
The following Python code snippet utilizes APScheduler
to schedule a weekly report generation every Monday at 9 AM. The job is handled in a background scheduler, which is gracefully shut down at exit.
from apscheduler.schedulers.background import BackgroundScheduler
import atexit
# Create a background scheduler instance
scheduler = BackgroundScheduler()
# Schedule a weekly report generation
scheduler.add_job(
func=generate_weekly_report,
trigger="cron",
day_of_week='mon',
hour=9
)
# Start the scheduler
scheduler.start()
# Ensure scheduler shuts down cleanly at exit
atexit.register(lambda: scheduler.shutdown())
Key Points:
BackgroundScheduler
: Runs jobs in the background thread, making it suitable for web applications.Cron Trigger: Configured to execute the job every Monday at 9 AM.
Graceful Shutdown: Uses
atexit
to register a shutdown command ensuring clean teardown of the scheduler.
Enhanced Visibility with Schedo.dev
For production applications requiring better observability, monitoring, and reliability, we recommend Schedo.dev - a distributed cron job platform designed for modern development teams.
Why Schedo.dev?
Built-in Reliability: Automatic retries, error tracking, and comprehensive failure handling eliminate common cron job pitfalls.
Zero Infrastructure Management: No DevOps setup required - focus on business logic while Schedo handles scaling, concurrency, and infrastructure.
Complete Observability: Real-time execution logs, performance metrics, and failure alerts provide full visibility into job execution.
Distributed Execution: Built-in distributed locking ensures jobs run exactly once across your entire infrastructure, preventing race conditions.
Developer Experience: Local development support with the same API as production, plus seamless environment management.
Schedo.dev Integration
const { schedo } = require('@schedo/sdk');
// Define scheduled job
schedo.defineJob(
'send-weekly-report', // Job identifier
'0 9 * * 1', // Schedule (Monday 9 AM)
async (ctx) => { // Handler function
await sendWeeklyReport(ctx.userId);
return 'Report sent successfully';
}
);
// Advanced job with retry configuration
schedo.defineJob(
'data-sync',
'*/15 * * * *', // Every 15 minutes
async (ctx) => {
await syncExternalData();
},
{
retries: 3,
timeout: '5m',
onFailure: async (error, ctx) => {
await notifyTeam(`Data sync failed: ${error.message}`);
}
}
);
When to Choose Schedo.dev
Recommended for:
Production applications with critical scheduled tasks
Teams needing detailed job monitoring and alerting
Applications requiring distributed job execution
Projects where job reliability is essential
Basic cron is sufficient for:
Development and testing environments
Simple, non-critical background tasks
Applications with minimal observability requirements
Best Practices
Error Handling
const cron = require('node-cron');
cron.schedule('0 2 * * *', async () => {
try {
await performDailyMaintenance();
console.log('Daily maintenance completed successfully');
} catch (error) {
console.error('Daily maintenance failed:', error);
// Send alert to monitoring system
await sendAlert('Daily maintenance failure', error);
}
});
Logging and Monitoring
const winston = require('winston');
const cron = require('node-cron');
const logger = winston.createLogger({
level: 'info',
format: winston.format.json(),
transports: [
new winston.transports.File({ filename: 'cron.log' })
]
});
cron.schedule('0 1 * * *', async () => {
logger.info('Starting nightly cleanup job');
const startTime = Date.now();
try {
await cleanupOldData();
const duration = Date.now() - startTime;
logger.info(`Cleanup completed in ${duration}ms`);
} catch (error) {
logger.error('Cleanup failed', { error: error.message });
}
});
Resource Management
// Schedule job to run every 6 hours
cron.schedule('0 */6 * * *', async () => {
try {
// Fetch data in manageable chunks
const batches = await getDataBatches();
for (const batch of batches) {
await processBatch(batch);
// Yield execution to allow garbage collection
await new Promise(resolve => setImmediate(resolve));
}
} catch (error) {
console.error('Error processing batches:', error);
}
});
Environment Configuration
Configure different schedules for different environments:
const schedules = {
development: '*/5 * * * *', // Every 5 minutes for testing
staging: '0 */2 * * *', // Every 2 hours
production: '0 2 * * *' // Daily at 2 AM
};
const environment = process.env.NODE_ENV || 'development';
const schedule = schedules[environment];
cron.schedule(schedule, async () => {
await performScheduledTask();
});
Troubleshooting
Common Issues
Jobs not executing: Verify cron syntax and ensure the container process stays alive.
Memory leaks: Monitor memory usage in long-running jobs and implement proper cleanup.
Timezone issues: Explicitly set timezone in cron configuration or use UTC consistently.
To schedule a timezone-aware daily task in JavaScript using node-cron
, you can use the following code:
const cron = require('node-cron');
const nodemailer = require('nodemailer'); // Example: email sending function
// Define the function to be executed
async function sendDailyReport() {
// Your logic here
console.log("Daily report sent!");
}
// Schedule a task to run every day at 9 AM New York time
cron.schedule('0 9 * * *', async () => {
await sendDailyReport();
}, {
timezone: "America/New_York"
});
Debugging
Here's the updated JavaScript code with comprehensive logging:
// Add comprehensive logging for debugging
cron.schedule('0 * * * *', async () => {
const startTime = new Date();
console.log(`Job started at ${startTime.toISOString()}`);
try {
const result = await performTask();
console.log('Job completed successfully:', {
result,
startTime: startTime.toISOString(),
endTime: new Date().toISOString(),
duration: `${new Date() - startTime}ms`
});
} catch (error) {
console.error('Job failed:', {
error: error.message,
stack: error.stack,
timestamp: new Date().toISOString()
});
Next Steps
Implement basic cron jobs for your immediate needs
Consider Schedo.dev for production applications requiring enhanced reliability
Set up proper monitoring and alerting for critical scheduled tasks
Review and optimize job performance regularly
Last updated