I’m going to train LoRa for Z Image Turbo. But I don’t like the base output (background and colors). I want to do some color adjustment steps with NAG and Detail Daemon.
Is the base output recommended for LoRa training, or should I train with the most optimal images for my goals?
How will the two approaches affect my results?
When using it as a base model for LoRA training, Base might be fundamentally more suitable than Turbo? But Turbo came out first and is more widespread… making the choice difficult.
A LoRA does not “fix” a model in the abstract. It learns to reproduce the distribution of whatever you train it on (subject, lighting, palette, background types, contrast curve, etc.). This is why dataset choice matters more than most hyperparameters.
Z-Image Turbo adds a second complication: it is a few-step distilled model that does not rely on classifier-free guidance (CFG) at inference and, in the official prompting guidance, does not use negative prompts at all. (Hugging Face) Turbo is typically run at ~9 steps with guidance scale 0.0.
That pushes more of the “look” (including background/color tese model prior, and makes classic “negative prompt fixes” weaker unless you use something like NAG.
Using images that are clean, consistent, and not heavily “styled” is often the most robust path.
In this case, training on “base-ish” outputs can be acceptable, because you are not trying to reprogram the global aesthetic—just teach a concept.
Then training on base-like images is not recommended for your goal, because:
For a look-changing objective, your training images should already reflect your desired look.
If you want different backgrounds and color behavior, training on your “optimal” images is the direct route.
But “optimal” should mean:
Why the caution: Detail Daemon explicitly notes that pushing it too far produces an oversharpened and/or HDR effect. (GitHub) If you train on images that already have that signature and then also apply Detail Daemon at inference, you can get “double enhancement.”
What improves
What stays the same
Typical failure mode
What improves
What gets worse
Typical failure mode
Train on images that are 80–90% your target look (good grading, but not extreme), then do the last 10–20% with NAG/Detail Daemon.
NAG is explicitly motivated by the problem that CFG-based negative guidance collapses in few-step regimes, and it restores effective negative-style control by operating in attention space. (arXiv)
Implication for you
Detail Daemon adjusts sigma/sampling behavior to enhance detail and can reduce unwanted blur, but can produce an oversharpen/HDR look if pushed. (GitHub)
Implication for you
The official Z-Image repo recommends for the Base model:
That is exactly the environment where you can:
Turbo is distilled for fast steps and “no CFG/negative prompts.” (Hugging Face)
A commonly reported issue is that LoRAs trained directly on Turbo can lose the acceleration behavior—images become blurry at “fast” settings while looking normal at slower settings. (Hugging Face)
If you must train on Turbo anyway, there are two established mitigation paths:
Train on Z-Image (Base) with a dataset that reflects your desired palette/background, then:
This leverages Base’s controllability and avoids Turbo’s distillation fragility. (GitHub)
Train for Turbo, but plan the pipeline up front:
If you want the palette/background sometimes but not always:
my_grade_style) and keep other captions content-focused.Use Detail Daemon primarily at inference; if you must use it in dataset creation, keep it mild and consistent to avoid baking in HDR/sharpen artifacts. (GitHub)
Thank you very much. The base model is very inconsistent and produces poor quality images, or perhaps I’m not getting the desired results because I’m applying the same prompting technique to Turbo as I did to the original. In Turbo, the face changed significantly when the scene changed, so I created one portrait and then converted it into 15 different portraits from various angles using Nano Banana. I trained a Face LoRa with it, and now I can produce images with more consistent faces, almost exactly as I want in Turbo. My goal, as you suggested, is to produce 40 images with different locations, lighting, and elements. Then I will train a new LoRa with it. Currently, with a 1:4 LR, 16 rank, no quantization, and BF16 output, I get a sweet spot of 2400-3000 at 3k stepping. I previously tested it with 4k stepping, and the sweet spots were 2200 and 2600. I want to continue with this approach for now.
However, I still have some questions. For 40 images, is rank 16 ideal, or should I choose 32 (I’ll choose FP32 for quality anyway)? But since all parameters affect each other, finding the optimal solution is really difficult. Will 3-4k step be enough when I go up to 40 images? Should I use 1280 base resolution or 1536 when creating the images? Because Ostris has a 1536 resolution option. If I increase the rank, I think I’ll need to adjust the LR to 5e-5.
To be honest, it’s a very challenging process for someone who wants to achieve perfection.
There’s no absolute right answer for training parameters by nature…
It becomes a matter of choosing what kind of LoRA (model) behavior is likely to emerge. And often, you can’t say for sure without actually going through fine-tuning…
The inconsistency in prompts between Turbo and Base seems to be because their prompt quirks are completely different…
Z-Image Turbo and Z-Image Base want different prompting and sampling behaviors:
So if you applied “Turbo-style prompting” (short prompt, no negatives, guidance ~0) to Base, Base can look sloppy or inconsistent; and if you apply “Base-style prompting” (strong negatives + higher guidance) to Turbo, Turbo can drift (including face changes) because it’s not operating how it was distilled to operate. (Hugging Face)
Your “face drifts when the scene changes” observation is very consistent with Turbo’s few-step behavior, and using a face LoRA to stabilize identity is a reasonable workaround.
Turbo tends to degrade when stacking multiple LoRAs. This is widely reported in community testing and discussion. (Hugging Face)
That means your plan is actually strategically good because it can produce a single “combined” LoRA (identity + environment handling) so you don’t have to stack a face LoRA + a scene/style LoRA at inference.
The main risk is feedback-loop learning: if your 40 images are all synthetic outputs from the same pipeline, the second LoRA can learn the pipeline’s artifacts and biases very strongly. The fix is simple: curate hard, and force real diversity (lighting, lenses, backgrounds, compositions, distances, expressions).
Rule of thumb:
Community experience is mixed: some people find rank 16 “enough,” others report rank 32+ helps photorealistic micro-detail. Treat those reports as anecdotal, but directionally useful. (Reddit)
Recommendation for your 40-image “multi-scene” run
Steps depend on batch size/repeats, but you can reason with “steps per image”:
If (roughly) batch=1 and repeats=1:
So if your “sweet spot” previously appeared around 2,400–3,000 steps on a smaller set, the same “training intensity” on 40 images will usually occur later (often ~6k–8k steps).
This matches common Turbo LoRA baselines that treat ~2,500–3,000 as a starting point for small datasets, not a hard universal number. (Hugging Face)
Practical recommendation
There are two separate issues here:
(A) What Turbo was “in-domain” for
A Tongyi discussion on resolution/latent sizing strongly implies staying within a domain like 768–1280 around the “1024 grid,” and points to predefined resolution choices used in their app. (Hugging Face)
(B) What tooling exposes
Many workflows expose resolution presets including 1024 / 1280 / 1536. (GitHub)
What this means for you
Recommendation
A very common safe band for diffusion LoRA LR is 1e-4 to 5e-5. (Hugging Face)
If you go from rank 16 → 32, reducing LR (e.g., 1e-4 → 5e-5) is a reasonable safety move because you’re increasing capacity and the training can become more “aggressive.”
If your training is actually happening in bf16/fp16 (which is typical in these toolchains), saving the LoRA as fp32 does not recover lost precision; it mostly stores already-rounded values in a larger container. (Reddit)
BF16 is generally considered a strong default for stability/efficiency, and FP32 is rarely the deciding factor for visual quality compared to dataset quality and stopping at the right step. (RunComfy)
Recommendation
Training directly on Turbo can break the acceleration behavior; this is why “training adapter / de-distillation” and “DistillPatch” approaches exist.
If your LoRA starts looking fine at 20–30 steps but falls apart at 8–9 steps, that’s a classic sign you need one of these “preserve acceleration” approaches. (Hugging Face)
Hyperparameters interact, but you can make the search manageable by fixing everything except one axis at a time:
If you try to change rank/LR/steps/resolution together, it becomes almost impossible to tell what helped.
Because you’re using generated images as training targets, you should deliberately inject variation so the second LoRA learns identity rather than “that one portrait’s hidden artifacts”:
This reduces memorization and improves the “face stays the same when the scene changes” property you care about.
I can’t thank you enough for your detailed explanations. I really appreciate it. I’m going to follow this step exactly now. Face Lora is working well, only the ear isn’t closing 
because it was visible in all the images, so I added this and similar things to the caption and will train it again. This way I’ll get rid of this problem. Even at strength 1, it gives the perfect exposure for the composition (except for the ear, which requires a very specific prompt to close :D).
So, Face Lora seems to be working hypothetically. Now, the next step is to create 40 images with real variety. Also, before doing that, I want to train Look Lora, as you said. I’ve collected many images from Unsplash: travel, gym, cafe, streets, etc. They are all very high quality and high resolution, but I think I’ll need to resize them a bit first.
So, what are your suggestions for Look Lora? Since there will be a lot of detail here, and it won’t be learning just one subject, I think 32 or a higher rank would be logical. I currently have 60 images. Do you think this is enough to train Look Lora? I’m leaving a screenshot below. What are my chances of getting sharp images with the LoRa tool I’ll be creating from these images? I specifically tried to select images where a person might be present.
I also trained a LoRa with the base model, now I’m going to try it with the turbo.
Claude also suggested that applying an img2img with ZIT to the Nano Banana outputs might be a good idea. But I think the images are good, high-quality, realistic, and consistent (except for the watermark, which I remove with another tool). Do you think it’s necessary?
Meanwhile, I think training can be done on the base model and then the turbo can be used. But as you said, 3k steps were insufficient; more are needed.
Using the capabilities and knowledge of high-end commercial AI for training open-source models is generally a good idea, not just for Nano Banana.
A Look LoRA is best at learning global visual biases: color palette, contrast curve, lighting mood, lens/render “feel,” background complexity preference. It is not a reliable way to “upgrade sharpness” beyond what the base model + sampling can already produce—unless the base model is already capable and your LoRA is mainly nudging it toward consistently sharp photographic structure.
With Z-Image Turbo, keep in mind:
This affects your “chances of sharp images” more than rank alone.
60 high-quality photos is enough for one coherent look (e.g., “clean editorial travel photography, natural skin, balanced contrast, slightly warm highlights”). If your set spans many unrelated aesthetics (harsh HDR city night + soft pastel cafe + gritty gym flash + cinematic teal/orange), the LoRA tends to learn a blurry average of the styles and may feel like it “does nothing” or introduces instability.
You intentionally included scenes where people might be present. That is good if your Look LoRA must preserve:
Rank is capacity. Higher rank can learn more variation, but it also makes it easier to learn unwanted correlations and artifacts. Diffusers’ LoRA config explicitly treats r (rank) as a core capacity parameter. (Hugging Face)
Rank 16 is usually the best first run for a Look LoRA with ~60 images.
Move to Rank 32 only if you can clearly describe a “capacity failure,” such as:
Why not start at 32?
With only 60 images, Rank 32 can more easily overfit on specific scene content (“this kind of street = this color cast”) instead of learning a transferable grade.
When dataset size increases, a fixed step count means fewer updates per image. So if 3k steps was your “sweet spot” on a smaller set, that does not transfer directly.
A simple planning heuristic:
Recommendation (works with your “sweet spot hunting” approach)
If you increase rank to 32, reducing LR (e.g., toward 5e-5) is a common stability move. (This aligns with typical LoRA guidance that higher capacity usually wants gentler updates.) (Hugging Face)
Reasons:
Only if:
Given your stated goal (look + backgrounds + colors, not microtexture replication), 1280 is the safer choice.
High, if these conditions hold:
You preserve Turbo’s fast-step behavior
Your training set is consistently sharp
Your captions don’t teach the wrong thing
LoRA can reduce sharpness if it breaks Turbo distillation or if it learns “soft” images as part of the look.
When it helps
When it hurts
Practical compromise
Example:
If you can’t summarize the look, the dataset is probably too mixed.
Turbo doesn’t use negative prompts, so any “don’t do X” must be phrased as positive constraints in your prompt. (Hugging Face)
Detail Daemon can help detail, but don’t overuse it in training data or you risk teaching the HDR/oversharpen signature. (GitHub)
Use one base template and change only {location}, {lighting}, {angle}:
{trigger_face}, {trigger_look}, photorealistic editorial portrait photo,
identity consistent face, natural skin texture, sharp focus on eyes,
balanced exposure, realistic shadows, clean color science, neutral white balance,
{lighting}, {angle}, {location},
high detail, clean edges, no text, no watermark, no logo, no extra faces
Turbo expects you to put constraints in the positive prompt, because negatives aren’t used. (Hugging Face)
Hopefully this title will rank highly on Google because we’ve discussed everything necessary throughout this process and provided very detailed links.
Great question! For LoRA training on Z Image Turbo, I’d recommend using the base output as your training foundation.
Why base output works better:
Alternative approach:
If you specifically need the color-adjusted outputs in your LoRA, consider:
The key insight is that LoRA learns the “difference” from the base model - so if you train on adjusted images, your LoRA will essentially be learning to correct back to base. Hope this helps!
Thanks for the reply. However, there are no IP adapters or similar solutions on Zit yet. I’m currently researching trigger words. Also, when exactly is LoRa considered to have learned? When it responds to every prompt that includes a female description and a trigger word? Or when it responds to a neutral prompt that uses a trigger word without specifying gender?