AWS Batch with Spot

TL;DR: AWS Batch schedules containerized jobs that don't need to run right now — ETL, transcoding, ML experiments, simulations — deciding where to run them, in what order, how to retry, and when to tear down. Pair it with Spot (spare EC2 at up to 90% off) and interruptions become the scheduler's problem, not yours: Batch reschedules reclaimed work on fresh capacity. The result is 70–85% off On-Demand for batch-shaped workloads, and the only thing you give up is the right to demand uninterrupted runtime.

The numbers

• Spot discount: typically 70–90% off — a c5.4xlarge at ~$0.68/hr On-Demand often runs $0.10–0.20/hr on Spot; 2-minute reclaim warning.
• Interruption rates sit below ~5%/hour on most pools; short jobs almost always finish first try.
• Fargate Spot runs tasks with zero instance management at ~70% off regular Fargate for jobs under 16 vCPU / 120 GB and no GPU.
• Field examples: a biotech genome pipeline went ~$30K → ~$5K/mo (75–85% off, per-chromosome checkpointing); a nightly image pipeline went $800 → $80/mo on Fargate Spot (~90%); a fintech Monte Carlo went $15K → <$3K/mo with SPOT_CAPACITY_OPTIMIZED + 15-min checkpoints.

Do this

1. Configure the compute environment for Spot with SPOT_CAPACITY_OPTIMIZED (or price-capacity-optimized) allocation — it steers toward the lowest live interruption risk, which matters as much as choosing Spot at all.
2. Diversify instance types across families and AZs so one pool's interruption doesn't take down the fleet.
3. Set min_vcpus = 0 — Batch auto-scales to zero on idle; a nonzero floor silently bills round-the-clock for warm capacity you rarely need.
4. Add a Spot→On-Demand fallback so jobs always run, paying On-Demand only when Spot capacity is genuinely unavailable.
5. Checkpoint jobs longer than ~1 hour to S3 — most frameworks (Spark, PyTorch, TensorFlow) have it built in; set the interval by the cost of losing that work (15–30 min for long HPC jobs).

Gotchas

• Checkpointing scales with job length: under 10 min needs none, 1–4 hours makes it nice-to-have, over 4 hours makes it mandatory — a 12-hour job with 30-min checkpoints rarely loses more than that to an interruption.
• Allocation strategy is a real lever — the default is not capacity-optimized; switching cut interruption rates meaningfully in field cases.
• Thin GPU Spot pools (newest accelerators) can have interruption rates high enough to outweigh the discount.
• The nonzero-min-vCPU leak is the most common cost mistake — default to zero.

Skip this if

• The job is latency-critical or user-facing (real-time scoring on every transaction) — a 2-minute Spot pause is visible to a customer; use regular Fargate or On-Demand EC2.
• It's a single-instance job that can't safely be retried (financial settlement, idempotency-sensitive writes) — the savings aren't worth the risk. For tasks too small to need a full Batch environment, Fargate Spot; to apply the same idea to non-batch fleets, Spot Instances.