How to Fix: Azure DevOps Pipeline Agent Timeout

The Scenario

Your Friday afternoon deployment to an AKS cluster via an Azure DevOps YAML pipeline has been running for 45 minutes. The pipeline was building a .NET 8 application with a long-running integration test suite. Suddenly, the job fails with a “The job running on agent exceeded the maximum execution time” error. The pipeline is set to timeout after 60 minutes, and you’re now blocked from shipping the hotfix.

Symptoms

• Pipeline job fails with: ##[error]The job running on agent <agent-name> exceeded the maximum execution time of 60 minutes.
• The job status shows “Cancelled” or “Failed” in Azure DevOps UI with a timeout icon.
• Logs stop abruptly at the last step before timeout, often during a long-running test, deployment, or build step.

Root Cause

Azure DevOps pipeline jobs have a default maximum timeout of 60 minutes for hosted agents (and no limit for self-hosted agents unless explicitly set). This timeout is enforced at the job level, not the individual task level. When a single step (e.g., a test suite, Docker build, or Terraform apply) runs longer than the job timeout minus the time spent on previous steps, the entire job is killed. The root cause is typically an unoptimized step that takes too long, not a network or agent connectivity issue.

Resolution (Step-by-Step)

1. Increase the job timeout in your pipeline YAML
   Add timeoutInMinutes at the job level to allow more time for the long-running step. For hosted agents, the maximum is 360 minutes (6 hours) for private projects and 60 minutes for public projects.

   jobs:
   - job: BuildAndDeploy
     timeoutInMinutes: 120  # Increase to 2 hours
     pool:
       vmImage: 'ubuntu-latest'
     steps:
       # ... your steps

2. If using a self-hosted agent, set the job timeout explicitly
   Self-hosted agents have no default timeout, but you should still set one to prevent runaway jobs.

   jobs:
   - job: Deploy
     timeoutInMinutes: 0  # 0 means no timeout (use with caution)
     pool:
       name: 'MySelfHostedPool'
     steps:
       # ... your steps

3. Identify and optimize the specific step causing the timeout
   Check the pipeline logs to find which step was running when the timeout occurred. Common culprits:

   • Long-running tests: Split tests into parallel jobs or use --filter to run critical tests first.
   • Docker builds: Use Docker layer caching and multi-stage builds.
   • Terraform apply: Break into smaller workspaces or use -target for incremental deployments.

   Add timeoutInMinutes to the specific task if needed:

   - task: DotNetCoreCLI@2
     displayName: 'Run integration tests'
     timeoutInMinutes: 30  # Task-level timeout
     inputs:
       command: test
       projects: '**/*Tests.csproj'

4. For hosted agents, switch to a larger VM SKU
   If the step is CPU or memory bound, using a windows-latest or macOS-latest agent might not help. Instead, use a custom VM image or self-hosted agent with more resources.

5. Use the cancelTimeoutInMinutes setting for graceful shutdown
   If your step needs cleanup before cancellation, set a cancellation timeout:

   jobs:
   - job: Deploy
     cancelTimeoutInMinutes: 5  # Allow 5 minutes for cleanup

Why This Sometimes Doesn’t Work

Increasing the job timeout is a band-aid, not a fix. If your pipeline consistently takes longer than expected, the underlying step (e.g., a test suite that grows over time) will eventually exceed any reasonable timeout. I’ve seen teams bump the timeout to 360 minutes only to have the pipeline fail at 359 minutes because a test was hanging. Always profile the step: run it locally with --diagnostics or add logging to see where time is being spent. Also, be aware that Azure DevOps hosted agents have a hard limit of 6 hours, so if your job needs more than that, you must use self-hosted agents or split the job into multiple pipeline runs.

Verification

Run a test pipeline with the increased timeout and monitor the job duration in Azure DevOps:

# Check the job duration via Azure CLI
az pipelines build show --id <build-id> --org <org> --project <project> --query "timeLine.records[?type=='Job'].{name: name, startTime: startTime, finishTime: finishTime}" -o table

Expected output: The job should complete without the timeout error, and the finishTime should be less than the timeoutInMinutes value.

Common Follow-up Questions

Q: Can I set a timeout for the entire pipeline, not just a job? A: Yes, use timeoutInMinutes at the pipeline level (under trigger: or schedules:), but this is rarely needed. Job-level timeouts are more granular and easier to debug.

Q: Why does my self-hosted agent still timeout even with timeoutInMinutes: 0? A: Check the agent’s configuration file (~/.agent). The agent itself has a agent.listenTimeout setting that defaults to 60 minutes. Set agent.listenTimeout=0 in the .agent file and restart the agent service.

Q: How do I debug a hanging task without waiting for timeout? A: Use the Azure DevOps REST API to cancel the job early, or add a condition: always() step at the end of your job that runs cleanup regardless of failure. For self-hosted agents, you can SSH into the agent machine and inspect running processes.

Prevent This in the Future

Monitor your pipeline durations over time with a tool like Better Stack to catch trends—like a test suite that grows 5% each sprint—before they hit the timeout limit. Proactive alerting on pipeline duration anomalies can save you from Friday afternoon firefights. Set up a dashboard that tracks the P95 duration of your critical pipelines and alerts you when it exceeds 80% of your configured timeout. Set up uptime monitoring and alerts with Better Stack