Score Millions of Rows Offline: SageMaker Batch Transform with Input Filtering and Join

Many production ML workloads are not interactive. Nightly churn scores, weekly demand forecasts and back-catalog document classification do not need a real-time endpoint that bills around the clock. SageMaker Batch Transform spins up instances, scores an S3 dataset, writes results back to S3 and shuts everything down. This tutorial shows how to run it well.

It assumes you already have a SageMaker model (a trained artifact plus inference container) and an execution role. For help choosing between inference types, see our comparison of real-time, serverless, async and batch.

1. Create the model and transformer

import sagemaker
from sagemaker.estimator import Estimator

session = sagemaker.Session()
bucket = session.default_bucket()

# Attach to a finished training job (or build a Model object from model_data)
estimator = Estimator.attach("my-xgboost-training-job")

transformer = estimator.transformer(
    instance_count=2,
    instance_type="ml.m5.xlarge",
    strategy="MultiRecord",        # batch many records per request
    assemble_with="Line",
    accept="text/csv",
    max_payload=6,                  # MB per request
    max_concurrent_transforms=4,    # parallel requests per instance
    output_path=f"s3://{bucket}/scores/2026-01-01/",
)

strategy="MultiRecord" packs as many records as fit into max_payload; SingleRecord sends one per request and is much slower. Keep max_concurrent_transforms near the number of worker processes your container runs.

2. Run the job with input and output filters

Tabular models usually need features only, but you want an ID next to each prediction. Batch Transform can select input columns and join the output with the original record using JSONPath expressions.

transformer.transform(
    data=f"s3://{bucket}/features/2026-01-01/",
    content_type="text/csv",
    split_type="Line",
    input_filter="$[1:]",      # drop column 0 (the record ID) before inference
    join_source="Input",        # append the prediction to the original row
    output_filter="$[0,-1]",   # keep the ID and the prediction only
    wait=True,
    logs=False,
)

The result is one output file per input file, with the ID and prediction on each line, ready for Athena or a downstream load. Note: if your input has a header row, strip it first, because Batch Transform treats every line as a record.

3. Tune throughput

• Instance count: files are distributed across instances, so you get no benefit from more instances than input files. Shard large inputs into many roughly equal files.
• Payload and concurrency: raise max_payload until memory or latency per request degrades, then raise concurrency until CPU or GPU saturates. Watch CPUUtilization and MemoryUtilization in the /aws/sagemaker/TransformJobs CloudWatch metrics.
• Timeouts: set invocations_timeout_in_seconds for slow models; the default can be too short for large payloads.
• Instance type: use GPU instances only if the model needs them. Many tabular and classical models are cheaper on CPU.

4. Handle failures

One malformed record should not kill a six-hour job. Set the MaxPayloadInMB conservatively and make your inference handler return a clear error for bad rows. Check the job status and failure reason programmatically:

import boto3
sm = boto3.client("sagemaker")
d = sm.describe_transform_job(TransformJobName=transformer.latest_transform_job.name)
print(d["TransformJobStatus"], d.get("FailureReason"))

For retries, rerun only the shards that failed rather than the whole dataset.

5. Schedule it

Wrap the job in a SageMaker Pipeline TransformStep, and trigger the pipeline with Amazon EventBridge Scheduler on your cadence. This gives you lineage, parameterised input paths, and a clean place to add a quality check step before results are published.

6. Cost notes

• You pay for instance time only while the job runs, plus S3 storage.
• Use Spot-friendly patterns elsewhere in the pipeline, but note Batch Transform itself uses on-demand instances.
• Compress or use Parquet upstream so that less time is spent reading input.
• Choose Batch Transform over an endpoint whenever the latency requirement is minutes or more.

When to choose something else

If individual requests are large (long audio, big documents) and need results within minutes, asynchronous inference is a better fit. If you need sub-second responses, use a real-time endpoint. For everything scheduled and bulk, Batch Transform is the simplest and cheapest option.

Need help moving a bulk scoring workload onto SageMaker? Get in touch with our team.