Autoscaling on Edge Site for AWS

Learn how to effectively deploy autoscaling in your infrastructure to optimize resource utilization and handle fluctuations in workload more efficiently.

Updated at January 19th, 2026

Table of Contents

Introduction How It Works System Architecture Processing Worker Groups Deployment Process Technical Features Intelligent Auto-Scaling Monitoring & Observability Maintenance Mode Security & Compliance Infrastructure Components Getting Started Prerequisites Next Steps Support & Resources Documentation Vionlabs Support

Introduction

Vionlabs Cloud Processing is an advanced video analysis system that automatically extracts metadata, features, and insights from your video catalog. The system deploys directly into your AWS environment, ensuring data security while providing enterprise-grade scalability and cost efficiency.

Key Benefits

Cost-Efficient - Pay only for actual processing time, with automatic scale-to-zero when idle
Fully Automated - No manual intervention required for scaling or resource management
Secure - Runs entirely within your AWS VPC, with no external data exposure
Rapid Deployment - Get up and running in under an hour
Scalable - From single videos to millions, the system adapts automatically

How It Works

System Architecture

Vionlabs Cloud Processing uses AWS Auto Scaling to create an elastic, on-demand processing infrastructure that automatically responds to your workload.

Idle State

When there's no processing to do, the system maintains zero EC2 instances.

Active Processing

When processing jobs arrive:

Vionlabs Cloud submits jobs to your AWS SQS queues
The system calculates required compute resources
EC2 instances launch automatically (within 2-3 minutes)
Each instance processes videos from your S3 bucket
Results are stored in your output S3 bucket
When complete, instances automatically terminate

⚠️ Everything is automated - you never need to manually start or stop instances.

Processing Worker Groups

The system organizes processing into two specialized worker groups, each with independent scaling:

Batch Group

Purpose: Regular feature extraction processing for your video catalog

Typical Scale: 5-50 instances

Use Case: Bulk processing, catalog analysis, scheduled jobs

Jitclip Group

Purpose: On-demand thumbnail and preview clip generation

Typical Scale: 1-5 instances

Use Case: Real-time preview generation, user-triggered requests

This separation ensures that high-volume batch processing doesn't interfere with responsive preview generation, and vice versa. Each group scales independently based on its workload.

Deployment Process

Getting started with Vionlabs Cloud Processing is straightforward:

Step 1	Share Your AWS Details Provide your AWS Region and Account ID to Vionlabs support. This allows us to share the processing AMI with your account.
Step 2	Receive Deployment Package Vionlabs sends you a .zip package containing Terraform scripts, configuration templates, and documentation, along with the AMI IDs for your region.
Step 3	Configure Fill in a simple configuration file with your AWS details (region, credentials, deployment prefix, and AMI IDs). Set your desired maximum instance counts for each worker group.
Step 4	Deploy with Terraform Run terraform apply to create all AWS infrastructure (Auto Scaling Groups, S3 buckets, SQS queues, IAM roles, etc.). Deployment takes approximately 10 minutes.
Step 5	Grant Input Bucket Access Attach an IAM policy to allow the processing instances to read videos from your input S3 bucket. A sample policy is provided in the README.
Done!	System Ready Send Terraform outputs to Vionlabs for final cloud-side configuration. Once confirmed, you're ready to process videos!

Technical Features

Intelligent Auto-Scaling

The system continuously monitors workload and automatically adjusts compute resources:

Scales from 0 to your configured maximum
Responds to workload within 2-3 minutes
Automatically terminates idle instances
No manual intervention required

Monitoring & Observability

Gain real-time visibility into your processing infrastructure:

CloudWatch Dashboards - Automatically created with key metrics
Capacity Metrics - Current, desired, and max instance counts
Queue Depth - Number of jobs waiting to be processed
System Health - Instance lifecycle and processing status

Maintenance Mode

A built-in system recovery mechanism for troubleshooting or updates:

Gracefully terminates all processing
Drains job queues
Returns system to known clean state
Useful for AMI updates or issue resolution

Security & Compliance

Data stays in your account - Videos and results never leave your AWS environment
VPC isolation - Processing runs within your private network
IAM-based access control - Principle of least privilege
Encryption at rest and in transit - S3 and EBS encryption supported

Infrastructure Components

The Terraform deployment creates the following AWS resources in your account:

Component	Purpose
Auto Scaling Groups	Two groups (batch and jitclip) manage EC2 instance lifecycle
S3 Buckets	Store processed outputs and extracted thumbnails/clips
SQS Queues	Distribute processing jobs and collect metrics/events
SNS Topics	Enable job submission and event notification
IAM Roles	Grant processing instances access to S3, SQS, SNS, ECR, CloudWatch
CloudWatch Dashboard	Real-time monitoring and metrics visualization

Note: Your input S3 bucket (where video files are stored) is not created by the deployment. You'll grant the processing system read access to your existing bucket via IAM policy.

Getting Started

Prerequisites

Before deployment, ensure you have:

AWS Account with administrative privileges
VPC with at least one subnet
Input S3 Bucket containing videos to process
Terraform v1.13.2 or later installed locally

Next Steps

Contact Vionlabs support to initiate onboarding
Review the README.txt in the deployment package for detailed technical instructions
Prepare your AWS environment (VPC, subnet, credentials)
Plan your scale (how many max instances for each worker group)

Support & Resources

Documentation

README.txt - Complete technical deployment guide (included in package)
Configuration Templates - Sample .tfvars files (included in package)
IAM Policy Examples - For input bucket access (included in package)

Vionlabs Support

Our team is here to help with:

Deployment assistance and troubleshooting
Configuration optimization
Scaling recommendations
AMI updates and maintenance