AI workflow optimization

Benchmark Before You Optimize: Turning AI Generation Speed Into a Measurable Loop

AI generation speed is easy to complain about and hard to improve. The work becomes tractable when the team stops tuning by instinct and starts measuring the actual phases of the workflow.

When an AI generation workflow is slow, the first instinct is to tweak the model. Lower the steps. Change the sampler. Try a different runtime. Quantize something. Batch more images. Batch fewer images. Turn on a flag that sounds like it should help.

Some of those changes may help. Some may make the system worse while making one run look faster. The hard part is not finding knobs. The hard part is knowing which knob changed which part of the workflow.

In a local FLUX.2 image-generation system, the early mental model was simple enough: the first warmup was expensive, each prompt had a large encoding or preparation cost, and each image had a sampling cost. That model was directionally useful, but it needed to be turned into a measurement loop before any optimization work could be trusted.

Speed Is Not One Number

"Generation took an hour" is a complaint, not a diagnosis. The same wall-clock time can hide very different systems.

One run might be slow because the runtime is cold and loading model weights. Another might be slow because every prompt pays a text encoding cost. Another might be slow because images are generated one at a time when the runtime could batch them. Another might be slow because final files are being written, uploaded, verified, and cleaned up through a storage pipeline.

Those are not the same problem. They should not get the same fix.

Before optimizing an AI workflow, split "slow" into phases the system can measure.

The Batch Shape Matters

A small implementation detail can change the performance model. In this system, the app submitted one ComfyUI prompt per generated text prompt, and each submission used batch_size = imagesPerPrompt. That means "2 prompts x 10 images" was not twenty independent submissions. It was two submissions, each producing a batch of ten images.

That matters because the prompt-level cost and the image-level cost scale differently. If prompt preparation is expensive, larger batches may help. If image sampling dominates, batching may do less than expected. If memory pressure rises with batch size, larger batches may become slower or less reliable.

In one local estimate, the assumed shape was roughly:

• initial runtime warmup: expensive, but ideally paid once
• per-prompt preparation: expensive and paid for each prompt submission
• per-image sampling: significant and paid for every generated image

With that model, a two-prompt, twenty-image job is not just "ten images times the image cost." It includes two prompt-level costs and twenty image-level costs. That kind of sanity check prevents teams from optimizing the wrong part of the run.

The Benchmark Loop

A useful benchmark loop should be small, repeatable, and boring. A good first test is not "run the whole production batch and see if it feels better." A better first test is a controlled job such as two prompts with two images each.

1. Start from a known runtime state.
2. Run the same prompts, seed, dimensions, steps, model mode, LoRA, and negative prompt.
3. Record timestamps for every meaningful phase.
4. Change exactly one variable.
5. Run the same job again.
6. Compare phase timings, not just total time.

This is not bureaucracy. It is how the team avoids fooling itself. Without a controlled loop, a faster run might be the result of a warm cache, a changed prompt, a skipped cleanup, lower image quality, a different batch shape, or a silent failure.

Measure the Phases

The benchmark should record enough information to answer where the time went. It does not need a new database schema to start. A local JSON file, queue logs, or structured terminal output is enough for an investigation phase, especially when the goal is to keep the work easy to discard if it proves unhelpful.

Once these phases are visible, the optimization conversation changes. The team no longer asks "why is it slow?" It asks "which phase is slow, and what is the smallest change that could improve that phase without breaking output quality or recovery?"

Change One Variable at a Time

The obvious variables are not always the best first variables. The right order depends on the measured bottleneck. Still, a disciplined generation benchmark usually tests these categories:

• batch shape: more images per prompt versus more prompt submissions
• step count: lower steps only if image quality remains acceptable
• runtime warmth: cold start, warm runtime, and post-cleanup behavior
• weight dtype: explicit loader settings, not misleading filenames
• reference handling: upload and encoding overhead for input images
• storage overhead: local file read, MinIO upload, DB writes, and verification

The key phrase is "one variable." If a run changes batch size, prompt wording, step count, cleanup policy, and storage behavior at once, it may produce a faster result. It will not produce knowledge.

Avoid False Wins

AI performance work has a special class of false wins. A workflow can become faster because it is doing less work, producing worse work, or failing to record what happened.

A lower step count might reduce detail. A smaller batch might look faster because it shifts cost into more submissions. A skipped cleanup might help on a production machine with dedicated GPUs but hurt a single-card development machine. A different runtime might promise throughput but introduce startup fragility. A fallback might turn a real generation failure into a fake success state.

The benchmark should therefore record not only timing, but also the exact prompts, settings, seeds, model mode, LoRA, output count, storage object keys, and any errors. Speed without provenance is not a useful engineering result.

Make the Loop Product-Visible

Early performance work can start in scripts, but the destination should be product-visible. If an operator has to ask an engineer what phase the system is in, the system is still too opaque.

Long-running AI work should show whether it is queued, loading, warming, encoding, sampling, saving, ingesting, cleaning up, stalled, failed, or complete. It should show elapsed time and the last useful event. It should make retries and repair actions explicit. It should not require someone to read raw runtime logs to know whether a job is alive.

This is why performance instrumentation and product UX are connected. The same phase timings that help engineers optimize also help users trust the system during a long generation run.