Building a LeetCode-Style Code Evaluator with Isolated VM

The isolated-vm Solution

The isolated-vm library addresses the fundamental security challenges of running untrusted JavaScript by creating true V8 isolates--independent JavaScript execution contexts that share nothing with the host process. Unlike Node.js built-in vm module which runs code in the same process with limited scope separation, isolated-vm creates complete isolation where user code cannot access the host's memory, filesystem, or network.

Key Advantages

• Memory limits: Prevent runaway code from consuming system resources
• Execution timeout: Stop infinite loops automatically
• Complete sandbox: Malicious code cannot affect the host system
• Data transfer: Structured cloning for safe cross-isolate communication

The library has been adopted by production systems including the Temporal Node.js SDK for workflow determinism, demonstrating its reliability for critical applications. Its maturity and active maintenance make it suitable for building production code evaluation systems that require robust security guarantees. Organizations building secure web applications increasingly rely on sandboxed execution environments for handling user-provided code and automation workflows.

For teams implementing AI-powered automation systems that process user code or scripts, proper sandboxing is essential. Our AI automation services incorporate secure code execution patterns to handle dynamic content processing safely.

Architecture Overview

A LeetCode-style code evaluator consists of several interconnected components working together to provide a seamless user experience. The frontend provides a code editor with syntax highlighting, language selection, and submission controls. The API layer accepts submissions, manages queues, and returns results. The execution engine runs code in isolated sandboxes with resource limits. The test case manager stores, versions, and serves test data. Finally, the result aggregator collects execution outcomes, compares them against expected results, and generates feedback.

System Components

Data Flow

1. User submits code through the API
2. Submission is validated and queued
3. Execution worker picks up the submission
4. Worker creates isolated VM, transfers code and test cases
5. Code runs with configured limits
6. Results captured, compared, and stored
7. User notified of verdict

This flow ensures no single component becomes a bottleneck. The queue-based approach provides natural durability--submissions aren't lost if execution workers temporarily fail. Building such distributed systems requires careful web development architecture planning to ensure scalability and reliability.

When designing complex evaluation pipelines, consider how automation can streamline testing and deployment. Our AI automation expertise helps organizations implement intelligent workflows that scale with demand.

Setting Up the Environment

Installing isolated-vm

The isolated-vm library requires native compilation, which means having appropriate build tools installed on your system. On Debian-based systems, install build-essential and python3. On macOS, Xcode command line tools are required. Windows users need Visual Studio with C++ build tools. The installation process compiles the native module against your Node.js version, creating a binary optimized for your runtime environment.

# Debian/Ubuntu
sudo apt-get install build-essential python3

# macOS
xcode-select --install

# Install the package
npm install isolated-vm

The library supports Node.js versions 14 and above, with best results on recent LTS versions. After installation, verify the module loads correctly by running a simple test script. If you encounter compilation errors, check that your build tools are up to date and that your Node.js version is compatible with the isolated-vm version you're installing.

For production deployments, consider using pre-built binaries or Docker containers with isolated-vm pre-installed to avoid build-time dependencies in your deployment pipeline. Some teams also use tools like pkg or nexe to create standalone executables that bundle isolated-vm, simplifying deployment across different environments.

Creating the Isolated Execution Environment

A V8 isolate is a complete, independent JavaScript execution environment with its own heap, garbage collector, and execution stack. When you create an isolate, you're essentially spawning a new JavaScript "universe" that has no connection to the host process except through explicit channels you define. This isolation is what makes running untrusted code safe.

const isolate = new ivm.Isolate({
 memoryLimit: 128 * 1024 * 1024, // 128MB limit
 enableAtomics: true,
 inspector: false
});

Memory limits should be generous enough for legitimate solutions while preventing denial-of-service attacks through memory exhaustion. A limit of 128MB works for most algorithmic problems, though some data-intensive tasks may need more. The key is to start conservative and adjust based on actual usage patterns.

Implementing secure sandbox environments requires deep expertise in Node.js development and system architecture. Partnering with experienced developers ensures proper security configurations from the start.

Building the Executor Class

Create a reusable executor class that encapsulates isolate creation, code compilation, execution, and cleanup. This abstraction makes the evaluation system easier to maintain and test. The class should handle common scenarios like successful execution, timeout, memory exceeded, and runtime errors.

class CodeExecutor {
 constructor(options = {}) {
 this.isolate = new ivm.Isolate({
 memoryLimit: options.memoryLimit || 128 * 1024 * 1024
 });
 this.context = this.isolate.createContext();
 this.timeout = options.timeout || 5000;
 }

 async execute(code, testCases) {
 const context = this.context;
 
 // Set up timeout
 const timeoutId = setTimeout(() => {
 throw new Error('Execution timeout');
 }, this.timeout);

 try {
 // Compile and run user code
 await context.eval(code);
 return result;
 } finally {
 clearTimeout(timeoutId);
 }
 }

 dispose() {
 this.isolate.dispose();
 }
}

The executor class should be created for each evaluation and disposed after use. This ensures clean separation between submissions and prevents state leakage. The disposal step is critical for resource management--failing to dispose isolates leads to memory leaks and eventual system failure. Each isolate has its own context, which is the global scope where code executes, and you control what objects and functions are available in this context, limiting what user code can access.

Building robust execution pipelines that handle code safely is a core competency of professional web development services. Expert teams implement proper lifecycle management and resource cleanup patterns to ensure long-term system reliability.

Security Best Practices

Defense in Depth

Security requires multiple layers of protection, not just isolated-vm's built-in isolation. Even with isolated-vm's strong isolation, some attack vectors remain. Code running in an isolate can still consume CPU cycles, allocate memory, or run for extended periods before hitting limits. Implement defense in depth by combining isolate isolation with additional security measures:

• Network isolation: Prevent code from making external requests
• Filesystem restrictions: Limit access to only necessary paths
• Resource limits: Prevent resource exhaustion attacks
• Audit logging: Track all submissions for investigation

Consider the principle of least privilege when designing what capabilities to expose in your sandbox. Start with minimal permissions and only add what's strictly necessary for evaluation. Each additional capability potentially expands the attack surface, so evaluate each addition carefully.

Preventing Common Vulnerabilities

Input Validation

All input from users--code, test case data, configuration--should be treated as potentially malicious. Validate submissions against schemas before processing, sanitize any user-provided strings to prevent injection attacks, and reject submissions containing obviously dangerous patterns.

function validateSubmission(submission) {
 // Check code size
 if (submission.code.length > MAX_CODE_SIZE) {
 throw new ValidationError('Code exceeds maximum size');
 }

 // Check for dangerous patterns
 const dangerousPatterns = [
 /process\./, // Access to process object
 /require\s*\(/, // Dynamic require
 /import\s*\(/, // Dynamic import
 /eval\s*\(/, // eval usage
 /Function\s*\(/ // Function constructor
 ];

 for (const pattern of dangerousPatterns) {
 if (pattern.test(submission.code)) {
 throw new ValidationError('Code contains prohibited patterns');
 }
 }

 return true;
}

Validation should be fast and comprehensive. Rejecting obviously invalid submissions early saves execution resources and reduces attack surface. However, validation should not be so restrictive that it prevents legitimate solutions--balance security with usability.

Security-focused web development practices incorporate multiple layers of validation and sandboxing to protect systems from malicious inputs while maintaining usability.

Performance Optimization

Connection Pooling

Creating new isolates for each submission has overhead. Implement connection pooling to reuse isolates across multiple evaluations when safe to do so. Pooled isolates can be "warmed up" with common dependencies pre-loaded, reducing per-submission latency. The pool size should match your concurrency requirements and available memory.

class IsolatePool {
 constructor(options) {
 this.maxSize = options.maxSize || 10;
 this.available = [];
 this.inUse = [];
 }

 async acquire() {
 if (this.available.length > 0) {
 const isolate = this.available.pop();
 this.inUse.push(isolate);
 return isolate;
 }

 if (this.inUse.length < this.maxSize) {
 const isolate = await this.createIsolate();
 this.inUse.push(isolate);
 return isolate;
 }

 return this.waitForAvailable();
 }

 release(isolate) {
 this.inUse = this.inUse.filter(i => i !== isolate);
 this.available.push(isolate);
 }
}

Pool management requires careful consideration of state isolation. Isolate state from previous evaluations could leak into subsequent ones, causing incorrect results. For highly security-sensitive applications, creating fresh isolates for each submission may be necessary despite the overhead.

Optimization Strategies

• Cache test cases by problem ID since they're reused across submissions
• Pre-warm isolates with common dependencies
• Batch similar evaluations to reduce isolate creation overhead
• Monitor queue depth and auto-scale workers based on demand

Horizontal Scaling

Design the system to scale horizontally by adding more execution workers. The queue-based architecture naturally supports this by allowing workers to pull from a shared queue. Add load balancing in front of the API layer to distribute requests across multiple instances. Use shared storage for submission data so any worker can process any submission.

Scaling considerations include queue depth monitoring to detect when workers can't keep up with submissions, auto-scaling policies based on queue depth or submission latency, and graceful degradation under extreme load. Consider implementing rate limiting at the API level to prevent sudden traffic spikes from overwhelming the system.

For organizations implementing scalable automation solutions, our AI automation services provide the infrastructure and expertise needed to build systems that grow with your user base.

Production Deployment Checklist

Monitoring Essentials

Production systems require comprehensive monitoring to detect issues before they impact users. Track these metrics:

• Submission throughput: Requests per second
• Execution latency: P50, P95, P99 times
• Queue depth: Submissions waiting to be processed
• Error rates: Failures by type
• Resource utilization: CPU, memory, isolate count

Error Handling

Handle various error conditions appropriately:

Implement dead letter queues for submissions that fail repeatedly. These failed submissions can be investigated for patterns--either bugs in the evaluation system or malicious attempts to break it.

Testing Strategy

Testing a code evaluator requires multiple approaches:

• Unit tests: Individual components (executor, validator)
• Integration tests: Complete evaluation pipeline
• Chaos testing: Worker crashes, network failures
• Security tests: Malicious code patterns

Create a test suite with submissions that cover various scenarios: correct solutions, incorrect solutions, edge cases, timeout cases, memory limit cases, and syntax errors. Include submissions designed to test security boundaries. Run this test suite regularly and before any deployment to catch regressions.

Deploying production systems requires professional web development expertise to ensure proper monitoring, error handling, and scaling capabilities are in place from day one.

Conclusion

Building a LeetCode-style code evaluator with isolated-vm requires careful attention to security, performance, and reliability. The isolated-vm library provides the foundation for secure code execution, but the surrounding architecture determines how well the system performs in production.

Key principles to remember:

1. Isolation at multiple levels: Don't rely on a single security mechanism
2. Defense in depth: Combine sandboxing with additional protections
3. Comprehensive monitoring: Detect issues before they impact users
4. Graceful degradation: Handle failures without system-wide collapse

Start with a simple implementation that works correctly, then add optimization and hardening as you scale. With these foundations in place, you can build a code evaluation system that safely executes user code while providing the responsive, reliable experience that developers expect from coding platforms.

If you're building a production code evaluation platform, consider how these capabilities integrate with your broader web development services. A secure, scalable evaluation system can be a powerful differentiator for educational platforms and technical recruitment tools.

For organizations looking to implement secure code execution as part of a larger automation strategy, our AI automation services can help you design and build systems that handle dynamic content safely while delivering exceptional user experiences.

Sources

1. LogRocket: Building a LeetCode-style code evaluator with isolated-vm - Comprehensive implementation guide
2. System Design Handbook: Design a Coding Platform Like LeetCode - System architecture and scaling considerations
3. GitHub: isolated-vm - Official library documentation and API reference
4. Node.js VM Module Documentation - Security warnings about vm module limitations
5. Endor Labs: The Perils of Running Untrusted JavaScript Code - Security analysis of running untrusted code