The Scenario
You’re deploying a containerized microservice to Azure Container Instances (ACI) via a CI/CD pipeline on a Friday afternoon. The deployment hangs for 2 minutes, then fails with a generic “Image pull failed” message in the Azure portal. You’ve used this exact image tag in dev earlier today.
Symptoms
- Azure portal shows container group status:
Failedwith message “Image pull failed” az container logsreturns nothing (container never started)az container show --query containers[0].instanceView.currentStateshows{"state": "Waiting", "detailStatus": "ImagePullBackOff"}
Root Cause
ACI cannot download the container image from the specified registry. This happens for one of four reasons: (1) incorrect registry credentials (most common with private registries like ACR, Docker Hub, or third-party registries), (2) the image tag doesn’t exist in the registry, (3) the registry endpoint is unreachable (DNS or network issue), or (4) you’ve hit Azure’s image pull rate limits (250 pulls per 6 hours per registry for free-tier Docker Hub). ACI pulls images on every container start, so even a previously working image can fail if credentials rotate or the image is deleted.
Resolution (Step-by-Step)
-
Verify the image exists and is accessible
# Test pull locally with same credentials az acr login --name <registry-name> docker pull <registry-name>.azurecr.io/myapp:v1.0.0 # Or for Docker Hub docker login docker pull myapp:v1.0.0 -
Check ACI registry credentials configuration
# If using Azure Container Registry (ACR) az container create \ --resource-group <rg> \ --name <aci-name> \ --image <registry>.azurecr.io/myapp:v1.0.0 \ --registry-login-server <registry>.azurecr.io \ --registry-username <client-id> \ --registry-password <password> # For ACR, use managed identity instead (recommended) az container create \ --resource-group <rg> \ --name <aci-name> \ --image <registry>.azurecr.io/myapp:v1.0.0 \ --acr-identity <identity-resource-id> -
Validate the registry URL and image tag
# Common mistakes: missing tag, wrong registry hostname # Correct format: <registry-server>/<image>:<tag> # For ACR: myregistry.azurecr.io/myapp:latest # For Docker Hub: docker.io/myusername/myapp:latest (docker.io is implicit) -
Check for network restrictions
# If using private VNet, ensure outbound connectivity to registry # Test DNS resolution nslookup myregistry.azurecr.io # Check NSG rules allow outbound HTTPS (443) to registry -
Handle Docker Hub rate limits
# Add Docker Hub credentials explicitly az container create \ --resource-group <rg> \ --name <aci-name> \ --image docker.io/myusername/myapp:v1.0.0 \ --registry-login-server docker.io \ --registry-username <docker-hub-username> \ --registry-password <docker-hub-token>
Why This Sometimes Doesn’t Work
The most common gotcha is that ACI caches registry credentials at the resource group level, not per container instance. If you’ve previously deployed a container with incorrect credentials to the same resource group, ACI may cache those bad credentials and reject subsequent valid pulls for up to 10 minutes. You must delete the failed container group entirely (az container delete) and wait for the cache to clear before redeploying. Also, managed identity-based pulls to ACR require the identity to have AcrPull role assignment, which can take 5+ minutes to propagate.
Verification
# Check container group status after fix
az container show \
--resource-group <rg> \
--name <aci-name> \
--query "containers[0].instanceView.currentState.state"
# Expected output: "Running"
# Also check events
az container show \
--resource-group <rg> \
--name <aci-name> \
--query "containers[0].instanceView.events"
# Look for "Pulled" event with image digest
Common Follow-up Questions
Q: Why does it work locally but fail in ACI? Your local Docker daemon caches images, so you’re pulling from cache. ACI always does a fresh pull. Also, local Docker uses your machine’s network and credentials, while ACI uses its own.
Q: How do I use a private registry other than ACR or Docker Hub?
Pass the full registry URL with --registry-login-server, plus --registry-username and --registry-password. For example, ghcr.io/myorg/myapp:latest with a GitHub Personal Access Token.
Q: Can I retry the pull automatically?
ACI retries up to 3 times with exponential backoff (30s, 60s, 120s). If all fail, the container group stays in Failed state and you must redeploy.
Prevent This in the Future
Proactive monitoring of container startup events and registry health can catch image pull failures before they block deployments. Many teams use Better Stack to track ACI deployment status, registry authentication errors, and image availability with real-time alerts. Set up uptime monitoring and alerts with Better Stack