The Scenario
You’re deploying a new service or scaling out an existing one in ECS on a Friday afternoon. The CI pipeline passes, but the service stays in ACTIVE with zero running tasks. After a few minutes, the deployment times out, and you see the dreaded “service
Symptoms
- ECS service events show:
service <service-name> was unable to place a task aws ecs describe-servicesshowsdeploymentswithpendingCount > 0butrunningCount = 0- The ECS agent logs on container instances (if using EC2 launch type) show
CannotInspectContainerErrororResourceInitializationError
Root Cause
The ECS scheduler can’t find a container instance that satisfies the task’s resource requirements (CPU, memory, ports) and placement constraints (AZ, instance type, capacity provider). This usually happens when:
- Resource exhaustion: All instances in the cluster are at capacity for CPU/memory/ports
- AZ mismatch: The task has a VPC subnet constraint that doesn’t match any available instance
- Capacity provider issues: Fargate or EC2 capacity providers have reached their limits
- Service limits: Account-level limits for tasks per service or per cluster are hit
Resolution (Step-by-Step)
-
Check the service events for the specific reason
aws ecs describe-services --cluster <cluster-name> --services <service-name> --query 'services[0].events[0]' -
Verify task definition resource requirements
aws ecs describe-task-definition --task-definition <task-def-arn> --query 'taskDefinition.containerDefinitions[0].{cpu: cpu, memory: memory, memoryReservation: memoryReservation}'Ensure
cpuandmemoryvalues don’t exceed the instance type’s capacity (e.g., at3.mediumhas 2 vCPU and 4 GB memory). -
Check available resources in the cluster
aws ecs describe-clusters --clusters <cluster-name> --include STATISTICSLook at
registeredContainerInstancesCountandrunningTasksCountto see if the cluster is full. -
Inspect container instance resources (EC2 launch type)
aws ecs describe-container-instances --cluster <cluster-name> --container-instances $(aws ecs list-container-instances --cluster <cluster-name> --query 'containerInstanceArns' --output text) --query 'containerInstances[].{id: ec2InstanceId, remainingCPU: remainingResources[?name==`CPU`].integerValue, remainingMemory: remainingResources[?name==`MEMORY`].integerValue}' -
Verify VPC and subnet configuration
aws ecs describe-services --cluster <cluster-name> --services <service-name> --query 'services[0].networkConfiguration'Ensure the subnets and security groups exist and have available IP addresses. For Fargate, check that the subnets have a route to the internet (or VPC endpoints for ECR/CloudWatch).
-
Check capacity provider status (if using Fargate)
aws ecs describe-capacity-providers --capacity-providers <capacity-provider-name>Verify the provider isn’t at
INACTIVEorUPDATINGstatus. -
Review service limits
aws ecs describe-services --cluster <cluster-name> --services <service-name> --query 'services[0].deployments[0].desiredCount'Compare against your account’s service quota for tasks per service (default 1,000 for most regions). Request a quota increase via the AWS Support Center if needed.
Why This Sometimes Doesn’t Work
Even if you have enough total CPU/memory in the cluster, the scheduler can fail due to port conflicts — if your task definition specifies hostPort (e.g., 80 or 443) and all instances already have that port bound. Also, AZ-based placement constraints can silently block placement if you have tasks spread across 3 AZs but only 2 have available instances. Always check aws ecs describe-container-instances for remainingResources per instance, not just cluster totals.
Verification
Run the following to confirm the service is now placing tasks:
aws ecs describe-services --cluster <cluster-name> --services <service-name> --query 'services[0].{status: status, runningCount: runningCount, pendingCount: pendingCount, events: events[0].message}'
Expected output: "runningCount": <desiredCount>, "pendingCount": 0, and no recent “unable to place a task” events.
Common Follow-up Questions
Q: Why does it work with Fargate but not EC2?
A: Fargate abstracts instance management, so resource exhaustion is less common. EC2 requires you to manage instance capacity — check if instances are at max containers or if ECS_ENABLE_TASK_ENI is enabled for awsvpc networking.
Q: The error says “no container instances found” but I have instances registered.
A: This usually means the task’s launch type (EC2 vs Fargate) doesn’t match the instances, or the instances are in a different cluster. Verify the cluster name and that instances have the correct ecs:cluster tag.
Q: Can I force ECS to place a task on a specific instance?
A: Yes, use aws ecs run-task with --started-by or --placement-constraints with distinctInstance:true, but this bypasses the scheduler’s optimization. Better to fix the underlying capacity issue.
Prevent This in the Future
Proactive monitoring of cluster resource utilization and task placement failures can catch capacity issues before they block deployments. Setting up alerts on ECS service events and container instance remaining resources helps your team respond before users are affected. Many teams use Set up uptime monitoring and alerts with Better Stack to consolidate ECS health checks, container logs, and incident response workflows in one place.