Model vs. Harness: What Makes a Good Image Generator

·

by

A blur of pink, blue and orange gradient light

Key takeaways

  • A model is the trained network that makes the pixels. The harness is everything else, meaning prompt rewriting, reference handling, editing tools, history, safety filters, pricing and the interface.

  • Public leaderboards rank models through bare API endpoints, so they say little about the product you'll use all day.

  • The same model can behave differently in two products. The maker's own support staff acknowledged that one GPT Image model refuses prompts through the API that ChatGPT accepts.

  • The harness can also hurt. A UC Berkeley study found that automatic prompt rewriting cut DALL-E 3's advantage over its predecessor by nearly 58%.

  • Judge a generator by cost per usable image, not by cost per generation or by a leaderboard rank.

Where the word comes from

"Harness" comes from the agent world. An agent-tooling company defined it in a March 2026 post as every piece of code, configuration and execution logic that isn't the model itself, and several AI labs now use it the same way for the prompts, tools and context management wrapped around a model. That company also said it moved its coding agent from the top 30 to the top 5 on a public benchmark by changing only the harness, though the post gives no scores.

Image products are a looser fit for the term, so treat it as a borrowed one. There is no published test that runs one image model through two products and scores the results. What exists is a pile of smaller, documented differences, and they point the same way.

What the model decides

The model decides the ceiling. It sets how well a prompt is followed, how clean the anatomy and lighting look, whether text in the image is spelled correctly and how well an edit leaves the rest of the picture alone.

Leaderboards measure this half. Artificial Analysis tests API endpoints with one prompt per comparison, a fixed 1024 by 1024 size and a fixed seed, with watermarks and safety checks switched off where it can. At the time of writing, GPT Image 2.6 Sunburst leads its text-to-image board at 1262, ahead of GPT Image 2.5 Sunburst at 1230. Nano Banana 2.1 sits at 1177 and Seedream 5.5 at 1135.

Those gaps are real. The first and tenth places are 127 points apart, so the model still matters a lot. But the boards drift quickly. Arena's image-edit board, updated on November 8, puts GPT Image 2.5 Sunburst above 2.6 Sunburst, which is the reverse of Artificial Analysis. The 2.6 score there is flagged as preliminary because few people had voted on it yet.

A leaderboard also uses short single prompts. It can't tell you how a tool handles a 20-image campaign, a typo in a headline or the fourth round of "keep the face, change the jacket."

What the harness decides

Think of the harness as seven jobs, each one a place where two products built on the same model can differ.

1. Prompt rewriting. Many products expand your prompt before the model sees it. The published cookbook for DALL-E 3 says GPT-4 rewrites prompts ahead of the model in the API and returns the result in a field called revised_prompt. Another generator's Magic Prompt feature does something similar, and its API turns it off when you send a structured JSON prompt instead of plain text.

2. Reference images. Products differ in how many references they accept, what they do with them and whether the setting is visible. Nano Banana 2 and Pro are reported to take up to 14 reference images, and ChatGPT's 2.6 update added reusable references.

3. Editing. A conversational edit in ChatGPT, an inpainting brush and a mask in an editor are different tools even when the same model sits underneath. Some apps add an edit model that takes several reference images plus inpainting and outpainting.

4. Consistency across images. Keeping a character or product the same across a set depends on how the product stores and reuses context, not only on the model.

5. Safety layers. These sit outside the model and differ by product. A community forum thread from August describes gpt-image-2 refusing prompts through the API with moderation set to low, while ChatGPT accepts them. The company's support replied in October, acknowledged the gap and didn't explain it.

6. Speed and cost. Hosts price the same model differently. One aggregator lists FLUX.2 dev at $0.025 per image on Replicate and $0.0084 on fal. Those figures come from a third party, so check the live pricing pages before you rely on them.

7. History and organization. Whether you can find last Tuesday's good result, compare versions and export at full size has nothing to do with the model.

When the harness gets in the way

More layers aren't automatically better. In a UC Berkeley experiment, 1,891 participants each wrote 10 prompts to match a target image. Automatic LLM rewriting cut DALL-E 3's advantage over DALL-E 2 by nearly 58%, and people who adapted their own prompts did better than the rewriter did.

The pattern shows up in other places too. A rewriting step makes prompts easier for beginners and harder for experts, because you can't see or control what the model actually received. Agent builders report the same decay. A behavior added to cover for an older model can turn into dead weight once the model improves.

So a good harness gives you the controls to see what happened and turn it off. A bad one hides the work.

How to judge a generator

Run the same short test on every tool you're considering.

  • Use your real prompts. Take five to ten from the last month. Include one with text in the image, one with a reference photo and one edit that should leave most of the picture untouched.

  • Count the retries. Log how many generations it takes to get one you'd ship. Divide the spend by the images that pass your review. That figure is cost per usable image.

  • Check what changed. Look for a way to see the rewritten prompt, the seed and the model version. If you can't, you can't repeat a result.

  • Test the edit loop. Ask for four sequential changes to one image and check whether the first three survive.

  • Read the rights terms. Commercial use, training and ownership rules vary by product, and the model maker's terms usually apply on top.

  • Check the exit. Make sure you can export full-size files and move your work somewhere else.

If two tools tie on those tests, pick the one whose model ranks higher. If they split, the one with the better edit loop usually wins for repeat work, and the one with the stronger model wins for one-off hero images.

Where each kind of tool fits

Kind of tool

Example

Strongest half

Watch for

Chat assistant

ChatGPT Images, Gemini

Harness (conversational editing, reusable references)

Rewriting you can't inspect

Model maker's own API

GPT Image and Nano Banana APIs

Model (raw access, token pricing)

You build the editing and history yourself

Hosted open models

FLUX 3 and Seedream on third-party hosts

Price and choice

Defaults and prices vary by host

Frequently asked questions

What is the difference between a model and a harness?

The model is the trained network that generates the image, and the harness is everything around it that shapes the request and the result. That includes prompt rewriting, reference handling, editing tools, safety filters, history and pricing.

Does the model or the harness matter more for image quality?

The model sets the ceiling and the harness decides how often you reach it. For a single hero image the model usually dominates, and for repeated work with edits and consistency the harness usually does.

Why does the same model give different results in different apps?

Products wrap the model differently. They may rewrite your prompt, apply different safety filters, cap reference images at different counts or use different default sizes and settings.

Are leaderboards reliable for choosing an image generator?

They're reliable for comparing models and unreliable for comparing products. Artificial Analysis tests bare API endpoints with defaults and safety checks switched off where possible, so it leaves out the editor, history and interface you'll use.

Can prompt rewriting make results worse?

Yes. A UC Berkeley study found that automatic rewriting reduced DALL-E 3's advantage over DALL-E 2 by nearly 58%, and that people adapting their own prompts did better. Look for a tool that shows the rewrite or lets you turn it off.

How do I compare the cost of two generators?

Divide your total spend by the number of images you'd actually use, not by the number you generated. A cheaper tool that needs five tries can cost more than a pricier one that needs two.