IC-LoRA

IC-LoRAs (In-Context LoRAs) are specialized adapters for LTX-2 that condition video generation on a reference input, like an existing video clip or a control signal like a depth map. Each adapter is trained for a specific task: structural control, VFX, video restoration, creative transforms, or quality enhancement. Unlike standard LoRAs that modify style globally, IC-LoRAs perform targeted, reference-driven operations while preserving the original scene’s identity.

Comparison: IC-LoRA vs LoRA

When to use which:

• Need style/effect: Standard LoRA
• Need structural control, video restoration, VFX, or creative transforms: IC-LoRA
• Need both: Combine them (IC-LoRA 0.5-1.0 + effect LoRA 0.5-0.8)

Available IC-LoRA Adapters

For a full list of official IC-LoRA adapters — including Union Control, Motion Control, Water Simulation, Deblurring, and more — see the IC-LoRA Adapters page.

Using IC-LoRAs in ComfyUI

ComfyUI provides the recommended workflow for IC-LoRA due to its visual node-based interface for styling a video.

Setup:

1. Install ComfyUI-LTXVideo custom nodes
2. Download IC-LoRA model from Hugging Face (choose ComfyUI-compatible .safetensors)
3. Place in ComfyUI/models/loras/ directory
4. Load one of the ic-lora workflows

Key nodes:

• LTX IC-LoRA Loader Model Only: Loads the IC-LoRA checkpoint and applies it to the model. Supports the new generation of latent downscale.
• LTX Add Video IC-LoRA Guide: Applies the reference input with configurable attention_strength and attention_mask. Used by most IC-LoRA workflows.
• LTX Add Video IC-LoRA Guide Advanced: Extended version of the guide node with mask-aware conditioning for In-Outpainting. Not needed for other IC-LoRAs.
• LTX Draw Tracks: Draws spline-based motion trajectories visually for use with the Motion Control IC-LoRA
• LTX Sparse Track Editor: Converts sparse keypoint positions into interpolated per-frame coordinate paths
• LTX Sampler: Generates video following the control signal
• Text Encoder: Provides prompt conditioning

Example workflow steps:

1. Load depth map or pose skeleton video
2. Apply IC-LoRA with strength 1.0 (control details here)
3. Provide a descriptive text prompt
4. Generate video with standard LTX parameters

Preparing Control Signals

Control-type IC-LoRAs (Union Control, Motion Control) require an extracted control signal — a depth map, pose skeleton, edge map, or motion trajectory. The quality of these signals directly impacts results.

Most other IC-LoRAs (VFX, utility, and creative types) take a reference video clip directly as input, with no preprocessing or signal extraction needed. For these, skip to Reference Video below.

Video Walkthrough: Control IC-LoRAs

This walkthrough covers depth, canny, and pose control workflows in LTX-2 — a good introduction to control-signal-based IC-LoRAs.

Depth Maps

Tools:

• Depthcrafter
• Blender/3D software - For synthetic depth renders

Best practices:

• Use consistent depth range across all frames
• Ensure smooth temporal transitions (avoid flickering)
• Match resolution to target generation resolution

Pose Skeletons

Tools:

• OpenPose - Multi-person pose estimation
• MediaPipe Pose - Fast single-person pose
• DWPose - High-quality pose estimation

Best practices:

• Extract poses at consistent frame rate matching target generation
• Ensure pose keypoints are detected reliably across all frames
• Handle occlusions gracefully (interpolate missing keypoints)
• Use skeleton visualization format (lines connecting keypoints)

Format: Typically 17-18 keypoints for body skeleton rendered as visual overlay

Canny Edges

Tools:

• OpenCV Canny edge detection
• PIL/Pillow image processing
• ComfyUI Canny preprocessor nodes

Best practices:

• Adjust threshold values to capture essential edges without noise
• Maintain consistent edge thickness across frames
• Blur input slightly before edge detection to reduce noise

Sparse Track Conditioning

Used with the Motion Control IC-LoRA. Instead of providing a per-frame visual signal (like depth or pose), you define motion by placing keypoints at specific frames. The system interpolates smooth spline trajectories between them, rendered as trails of circles that the IC-LoRA follows.

Tools:

• LTX Draw Tracks node (ComfyUI) — visual canvas for drawing motion paths
• LTX Sparse Track Editor node (ComfyUI) — fine-tune keypoint positions and timing

Best practices:

• Start with 3-4 keypoints per track; add more only if the interpolated path doesn’t match your intent
• Keep trajectories physically plausible — sudden direction changes produce less natural results
• Match the track canvas resolution to your generation resolution

Reference Video

Most IC-LoRAs outside the Control category — including VFX (Water Simulation), utility (Colorization, Decompression, Deblurring), and creative types (Day to Night, Instant Shave, Cross-Eyed) — condition directly on a reference video clip. No signal extraction, preprocessing, or external tools are needed.

Best practices:

• Feed the reference video directly into the IC-LoRA workflow as-is
• Match the reference clip’s resolution and frame rate to your generation settings (25 fps is standard for most adapters)
• Some adapters use a trigger phrase prepended to the prompt (e.g. DEBLUR, REMOVEBEARD, COLORIZE, ADD WATER, ENHANCE QUALITY) — check the adapter’s README file for the correct trigger

IC-LoRA Parameters

Control Strength

LTX-2.3 introduced full strength control for IC-LoRAs through two complementary parameters:

attention_strength — Global strength scalar (0.0 to 1.0)

Controls overall IC-LoRA influence on generation. This works by scaling the cross-attention scores between the conditioning signal tokens and the noisy latents.

• 1.0 — Full adherence to the control signal (default, matches previous behavior)
• 0.5 — Balanced blend of control signal and free generation
• 0.0 — Control signal is completely ignored

attention_mask — Spatial or spatiotemporal mask (optional)

An optional mask that provides region-level control over where the IC-LoRA takes effect. The mask is multiplied by attention_strength to produce the final per-region strength.

• Spatial mask (H×W): Apply control to specific areas of each frame (e.g., apply depth control only to the foreground)
• Spatiotemporal mask (T×H×W): Vary control across both space and time (e.g., gradually introduce pose control over the first 2 seconds)
• Values range from 0.0 (no control) to 1.0 (full control)
• When omitted, uniform full strength is applied everywhere

How it works under the hood:

The strength control operates at the attention layer level. During cross-attention between the noisy video latents and the IC-LoRA conditioning tokens, the attention scores are scaled by attention_strength × attention_mask. This means the control signal’s influence can be precisely dialed in globally, regionally, or both.

Practical examples:

• Soft depth guidance: Set attention_strength to 0.6 for depth IC-LoRA to get approximate spatial structure while allowing the model more creative freedom
• Foreground-only pose control: Use a spatial attention_mask that’s 1.0 on the character region and 0.0 on the background
• Gradual control fade-in: Use a spatiotemporal mask that ramps from 0.0 to 1.0 over the first 8 frames
• Multi-region control: Combine with spatial masks to apply different IC-LoRA strengths to different parts of the frame

Resolution and Frame Rate

• Match control signal resolution to generation resolution
• Control FPS should match target generation FPS
• For best results: 704x1216 at 24-30 FPS
• IC-LoRAs work at various resolutions but quality depends on control signal clarity

Training Custom IC-LoRAs

Create your own IC-LoRA control adapters using the LTX-Video-Trainer.

Best Practices

Control Signal Quality

• Use high-quality control extraction tools
• Ensure temporal consistency (smooth transitions between frames)
• Match control resolution to generation resolution
• Pre-process control signals to remove noise and artifacts

Prompt Alignment

• For control-type IC-LoRAs, describe visual style, not control type (“ornate architecture” not “depth map shows…”)
• For IC-LoRAs with a trigger phrase, prepend it to the prompt before any scene description
• Align prompt with the reference input’s motion and composition
• Be specific about materials, lighting, and atmosphere
• Avoid contradicting the structure or content of the reference

Performance Optimization

• IC-LoRAs add minimal overhead (less than 10% compute)
• Works with FP8 quantized models
• Compatible with distilled models for faster generation

Quality Validation

• Always test IC-LoRA with simple control signals first
• Verify control is being respected before complex generations
• Compare with and without IC-LoRA to assess control strength
• Iterate on control signal quality before increasing generation complexity

IC-LoRA Troubleshooting

Control Signal Not Being Followed

Symptoms: Generated video ignores control structure

Solutions:

• Verify IC-LoRA is loaded correctly (check adapter name)
• Check attention_strength value — if set below 1.0, control will be proportionally weaker
• If using attention_mask, verify it covers the intended regions (mask values of 0.0 will disable control in those areas)
• Check control signal format matches expected input (resolution, channels)
• Validate control signal has sufficient contrast/detail
• Verify model compatibility (IC-LoRA version matches base model)

Temporal Flickering or Inconsistency

Symptoms: Unstable motion, frame-to-frame inconsistencies

Solutions:

• Smooth control signal temporal transitions (use interpolation)
• Increase inference steps (try 40-50 instead of 30)
• Ensure control signal FPS matches generation FPS
• Apply temporal filtering to control signal before generation
• Check for abrupt changes in control signal values

Poor Quality or Artifacts

Symptoms: Visual artifacts, degraded quality, unwanted textures

Solutions:

• Improve control signal quality (better extraction tools)
• Ensure control signal resolution is adequate
• Adjust prompt to better describe desired style
• Check that control signal doesn’t have noise or compression artifacts

Control Too Strong or Rigid

Symptoms: Output looks constrained, lacks natural variation

Solutions:

• Reduce attention_strength to 0.5-0.8 for a softer control effect
• Use a spatial attention_mask to limit control to specific regions
• Adjust your control signal to be less restrictive (e.g., lighter edges)
• Use more flexible prompts that allow style variation
• Consider if standard LoRA might be better for your use case

Memory Issues

Symptoms: Out of memory errors during generation

Solutions:

• Use FP8 quantized models to reduce VRAM
• Reduce generation resolution
• Process control signals in smaller batches

Resources

• IC-LoRA Adapters — All official IC-LoRA adapters with checkpoints, compatibility, and workflows
• LTX-Video-Trainer — Train custom IC-LoRA control adapters