DiffusionNFT/model/PrefRestorePipeline_model_structure.txt · Pref-Restoration

====================================================================================================
PrefRestorePipeline Model Structure Detailed Analysis
====================================================================================================

1. Basic Information
--------------------------------------------------
Model Path: /data/phd/yaozhengjian/zjYao_Exprs/BLIP-3o-next/Face-Restoration_FFHQ_VAE_Step3_scaling/checkpoint-108000
Device: cuda:0
Data Type: torch.bfloat16
Main Model Type: blip3oQwenForInferenceLMVAE
Tokenizer Type: Qwen2TokenizerFast
Processor Type: SiglipImageProcessor

2. Model Hierarchy Structure
--------------------------------------------------
: blip3oQwenForInferenceLMVAE
model: blip3oQwenModel
model.embed_tokens: Embedding
model.layers: ModuleList
model.layers.0: Qwen3DecoderLayer
model.layers.0.self_attn: Qwen3Attention
model.layers.0.self_attn.q_proj: Linear
model.layers.0.self_attn.k_proj: Linear
model.layers.0.self_attn.v_proj: Linear
model.layers.0.self_attn.o_proj: Linear
model.layers.0.self_attn.q_norm: Qwen3RMSNorm
model.layers.0.self_attn.k_norm: Qwen3RMSNorm
model.layers.0.mlp: Qwen3MLP
model.layers.0.mlp.gate_proj: Linear
model.layers.0.mlp.up_proj: Linear
model.layers.0.mlp.down_proj: Linear
model.layers.0.mlp.act_fn: SiLU
model.layers.0.input_layernorm: Qwen3RMSNorm
model.layers.0.post_attention_layernorm: Qwen3RMSNorm
model.layers.1: Qwen3DecoderLayer
model.layers.1.self_attn: Qwen3Attention
model.layers.1.self_attn.q_proj: Linear
model.layers.1.self_attn.k_proj: Linear
model.layers.1.self_attn.v_proj: Linear
model.layers.1.self_attn.o_proj: Linear
model.layers.1.self_attn.q_norm: Qwen3RMSNorm
model.layers.1.self_attn.k_norm: Qwen3RMSNorm
model.layers.1.mlp: Qwen3MLP
model.layers.1.mlp.gate_proj: Linear
model.layers.1.mlp.up_proj: Linear
model.layers.1.mlp.down_proj: Linear
model.layers.1.mlp.act_fn: SiLU
model.layers.1.input_layernorm: Qwen3RMSNorm
model.layers.1.post_attention_layernorm: Qwen3RMSNorm
model.layers.2: Qwen3DecoderLayer
model.layers.2.self_attn: Qwen3Attention
model.layers.2.self_attn.q_proj: Linear
model.layers.2.self_attn.k_proj: Linear
model.layers.2.self_attn.v_proj: Linear
model.layers.2.self_attn.o_proj: Linear
model.layers.2.self_attn.q_norm: Qwen3RMSNorm
model.layers.2.self_attn.k_norm: Qwen3RMSNorm
model.layers.2.mlp: Qwen3MLP
model.layers.2.mlp.gate_proj: Linear
model.layers.2.mlp.up_proj: Linear
model.layers.2.mlp.down_proj: Linear
model.layers.2.mlp.act_fn: SiLU
model.layers.2.input_layernorm: Qwen3RMSNorm
model.layers.2.post_attention_layernorm: Qwen3RMSNorm
model.layers.3: Qwen3DecoderLayer
model.layers.3.self_attn: Qwen3Attention
model.layers.3.self_attn.q_proj: Linear
model.layers.3.self_attn.k_proj: Linear
model.layers.3.self_attn.v_proj: Linear
model.layers.3.self_attn.o_proj: Linear
model.layers.3.self_attn.q_norm: Qwen3RMSNorm
model.layers.3.self_attn.k_norm: Qwen3RMSNorm
model.layers.3.mlp: Qwen3MLP
model.layers.3.mlp.gate_proj: Linear
model.layers.3.mlp.up_proj: Linear
model.layers.3.mlp.down_proj: Linear
model.layers.3.mlp.act_fn: SiLU
model.layers.3.input_layernorm: Qwen3RMSNorm
model.layers.3.post_attention_layernorm: Qwen3RMSNorm
model.layers.4: Qwen3DecoderLayer
model.layers.4.self_attn: Qwen3Attention
model.layers.4.self_attn.q_proj: Linear
model.layers.4.self_attn.k_proj: Linear
model.layers.4.self_attn.v_proj: Linear
model.layers.4.self_attn.o_proj: Linear
model.layers.4.self_attn.q_norm: Qwen3RMSNorm
model.layers.4.self_attn.k_norm: Qwen3RMSNorm
model.layers.4.mlp: Qwen3MLP
model.layers.4.mlp.gate_proj: Linear
model.layers.4.mlp.up_proj: Linear
model.layers.4.mlp.down_proj: Linear
model.layers.4.mlp.act_fn: SiLU
model.layers.4.input_layernorm: Qwen3RMSNorm
model.layers.4.post_attention_layernorm: Qwen3RMSNorm
model.layers.5: Qwen3DecoderLayer
model.layers.5.self_attn: Qwen3Attention
model.layers.5.self_attn.q_proj: Linear
model.layers.5.self_attn.k_proj: Linear
model.layers.5.self_attn.v_proj: Linear
model.layers.5.self_attn.o_proj: Linear
model.layers.5.self_attn.q_norm: Qwen3RMSNorm
model.layers.5.self_attn.k_norm: Qwen3RMSNorm
model.layers.5.mlp: Qwen3MLP
model.layers.5.mlp.gate_proj: Linear
model.layers.5.mlp.up_proj: Linear
model.layers.5.mlp.down_proj: Linear
model.layers.5.mlp.act_fn: SiLU
model.layers.5.input_layernorm: Qwen3RMSNorm
model.layers.5.post_attention_layernorm: Qwen3RMSNorm
model.layers.6: Qwen3DecoderLayer
model.layers.6.self_attn: Qwen3Attention
model.layers.6.self_attn.q_proj: Linear
model.layers.6.self_attn.k_proj: Linear
model.layers.6.self_attn.v_proj: Linear
model.layers.6.self_attn.o_proj: Linear
model.layers.6.self_attn.q_norm: Qwen3RMSNorm
model.layers.6.self_attn.k_norm: Qwen3RMSNorm
model.layers.6.mlp: Qwen3MLP
model.layers.6.mlp.gate_proj: Linear
model.layers.6.mlp.up_proj: Linear
model.layers.6.mlp.down_proj: Linear
model.layers.6.mlp.act_fn: SiLU
model.layers.6.input_layernorm: Qwen3RMSNorm
model.layers.6.post_attention_layernorm: Qwen3RMSNorm
model.layers.7: Qwen3DecoderLayer
model.layers.7.self_attn: Qwen3Attention
model.layers.7.self_attn.q_proj: Linear
model.layers.7.self_attn.k_proj: Linear
model.layers.7.self_attn.v_proj: Linear
model.layers.7.self_attn.o_proj: Linear
model.layers.7.self_attn.q_norm: Qwen3RMSNorm
model.layers.7.self_attn.k_norm: Qwen3RMSNorm
model.layers.7.mlp: Qwen3MLP
model.layers.7.mlp.gate_proj: Linear
model.layers.7.mlp.up_proj: Linear
model.layers.7.mlp.down_proj: Linear
model.layers.7.mlp.act_fn: SiLU
model.layers.7.input_layernorm: Qwen3RMSNorm
model.layers.7.post_attention_layernorm: Qwen3RMSNorm
model.layers.8: Qwen3DecoderLayer
model.layers.8.self_attn: Qwen3Attention
model.layers.8.self_attn.q_proj: Linear
model.layers.8.self_attn.k_proj: Linear
model.layers.8.self_attn.v_proj: Linear
model.layers.8.self_attn.o_proj: Linear
model.layers.8.self_attn.q_norm: Qwen3RMSNorm
model.layers.8.self_attn.k_norm: Qwen3RMSNorm
model.layers.8.mlp: Qwen3MLP
model.layers.8.mlp.gate_proj: Linear
model.layers.8.mlp.up_proj: Linear
model.layers.8.mlp.down_proj: Linear
model.layers.8.mlp.act_fn: SiLU
model.layers.8.input_layernorm: Qwen3RMSNorm
model.layers.8.post_attention_layernorm: Qwen3RMSNorm
model.layers.9: Qwen3DecoderLayer
model.layers.9.self_attn: Qwen3Attention
model.layers.9.self_attn.q_proj: Linear
model.layers.9.self_attn.k_proj: Linear
model.layers.9.self_attn.v_proj: Linear
model.layers.9.self_attn.o_proj: Linear
model.layers.9.self_attn.q_norm: Qwen3RMSNorm
model.layers.9.self_attn.k_norm: Qwen3RMSNorm
model.layers.9.mlp: Qwen3MLP
model.layers.9.mlp.gate_proj: Linear
model.layers.9.mlp.up_proj: Linear
model.layers.9.mlp.down_proj: Linear
model.layers.9.mlp.act_fn: SiLU
model.layers.9.input_layernorm: Qwen3RMSNorm
model.layers.9.post_attention_layernorm: Qwen3RMSNorm
model.layers.10: Qwen3DecoderLayer
model.layers.10.self_attn: Qwen3Attention
model.layers.10.self_attn.q_proj: Linear
model.layers.10.self_attn.k_proj: Linear
model.layers.10.self_attn.v_proj: Linear
model.layers.10.self_attn.o_proj: Linear
model.layers.10.self_attn.q_norm: Qwen3RMSNorm
model.layers.10.self_attn.k_norm: Qwen3RMSNorm
model.layers.10.mlp: Qwen3MLP
model.layers.10.mlp.gate_proj: Linear
model.layers.10.mlp.up_proj: Linear
model.layers.10.mlp.down_proj: Linear
model.layers.10.mlp.act_fn: SiLU
model.layers.10.input_layernorm: Qwen3RMSNorm
model.layers.10.post_attention_layernorm: Qwen3RMSNorm
model.layers.11: Qwen3DecoderLayer
model.layers.11.self_attn: Qwen3Attention
model.layers.11.self_attn.q_proj: Linear
model.layers.11.self_attn.k_proj: Linear
model.layers.11.self_attn.v_proj: Linear
model.layers.11.self_attn.o_proj: Linear
model.layers.11.self_attn.q_norm: Qwen3RMSNorm
model.layers.11.self_attn.k_norm: Qwen3RMSNorm
model.layers.11.mlp: Qwen3MLP
model.layers.11.mlp.gate_proj: Linear
model.layers.11.mlp.up_proj: Linear
model.layers.11.mlp.down_proj: Linear
model.layers.11.mlp.act_fn: SiLU
model.layers.11.input_layernorm: Qwen3RMSNorm
model.layers.11.post_attention_layernorm: Qwen3RMSNorm
model.layers.12: Qwen3DecoderLayer
model.layers.12.self_attn: Qwen3Attention
model.layers.12.self_attn.q_proj: Linear
model.layers.12.self_attn.k_proj: Linear
model.layers.12.self_attn.v_proj: Linear
model.layers.12.self_attn.o_proj: Linear
model.layers.12.self_attn.q_norm: Qwen3RMSNorm
model.layers.12.self_attn.k_norm: Qwen3RMSNorm
model.layers.12.mlp: Qwen3MLP
model.layers.12.mlp.gate_proj: Linear
model.layers.12.mlp.up_proj: Linear
model.layers.12.mlp.down_proj: Linear
model.layers.12.mlp.act_fn: SiLU
model.layers.12.input_layernorm: Qwen3RMSNorm
model.layers.12.post_attention_layernorm: Qwen3RMSNorm
model.layers.13: Qwen3DecoderLayer
model.layers.13.self_attn: Qwen3Attention
model.layers.13.self_attn.q_proj: Linear
model.layers.13.self_attn.k_proj: Linear
model.layers.13.self_attn.v_proj: Linear
model.layers.13.self_attn.o_proj: Linear
model.layers.13.self_attn.q_norm: Qwen3RMSNorm
model.layers.13.self_attn.k_norm: Qwen3RMSNorm
model.layers.13.mlp: Qwen3MLP
model.layers.13.mlp.gate_proj: Linear
model.layers.13.mlp.up_proj: Linear
model.layers.13.mlp.down_proj: Linear
model.layers.13.mlp.act_fn: SiLU
model.layers.13.input_layernorm: Qwen3RMSNorm
model.layers.13.post_attention_layernorm: Qwen3RMSNorm
model.layers.14: Qwen3DecoderLayer
model.layers.14.self_attn: Qwen3Attention
model.layers.14.self_attn.q_proj: Linear
model.layers.14.self_attn.k_proj: Linear
model.layers.14.self_attn.v_proj: Linear
model.layers.14.self_attn.o_proj: Linear
model.layers.14.self_attn.q_norm: Qwen3RMSNorm
model.layers.14.self_attn.k_norm: Qwen3RMSNorm
model.layers.14.mlp: Qwen3MLP
model.layers.14.mlp.gate_proj: Linear
model.layers.14.mlp.up_proj: Linear
model.layers.14.mlp.down_proj: Linear
model.layers.14.mlp.act_fn: SiLU
model.layers.14.input_layernorm: Qwen3RMSNorm
model.layers.14.post_attention_layernorm: Qwen3RMSNorm
model.layers.15: Qwen3DecoderLayer
model.layers.15.self_attn: Qwen3Attention
model.layers.15.self_attn.q_proj: Linear
model.layers.15.self_attn.k_proj: Linear
model.layers.15.self_attn.v_proj: Linear
model.layers.15.self_attn.o_proj: Linear
model.layers.15.self_attn.q_norm: Qwen3RMSNorm
model.layers.15.self_attn.k_norm: Qwen3RMSNorm
model.layers.15.mlp: Qwen3MLP
model.layers.15.mlp.gate_proj: Linear
model.layers.15.mlp.up_proj: Linear
model.layers.15.mlp.down_proj: Linear
model.layers.15.mlp.act_fn: SiLU
model.layers.15.input_layernorm: Qwen3RMSNorm
model.layers.15.post_attention_layernorm: Qwen3RMSNorm
model.layers.16: Qwen3DecoderLayer
model.layers.16.self_attn: Qwen3Attention
model.layers.16.self_attn.q_proj: Linear
model.layers.16.self_attn.k_proj: Linear
model.layers.16.self_attn.v_proj: Linear
model.layers.16.self_attn.o_proj: Linear
model.layers.16.self_attn.q_norm: Qwen3RMSNorm
model.layers.16.self_attn.k_norm: Qwen3RMSNorm
model.layers.16.mlp: Qwen3MLP
model.layers.16.mlp.gate_proj: Linear
model.layers.16.mlp.up_proj: Linear
model.layers.16.mlp.down_proj: Linear
model.layers.16.mlp.act_fn: SiLU
model.layers.16.input_layernorm: Qwen3RMSNorm
model.layers.16.post_attention_layernorm: Qwen3RMSNorm
model.layers.17: Qwen3DecoderLayer
model.layers.17.self_attn: Qwen3Attention
model.layers.17.self_attn.q_proj: Linear
model.layers.17.self_attn.k_proj: Linear
model.layers.17.self_attn.v_proj: Linear
model.layers.17.self_attn.o_proj: Linear
model.layers.17.self_attn.q_norm: Qwen3RMSNorm
model.layers.17.self_attn.k_norm: Qwen3RMSNorm
model.layers.17.mlp: Qwen3MLP
model.layers.17.mlp.gate_proj: Linear
model.layers.17.mlp.up_proj: Linear
model.layers.17.mlp.down_proj: Linear
model.layers.17.mlp.act_fn: SiLU
model.layers.17.input_layernorm: Qwen3RMSNorm
model.layers.17.post_attention_layernorm: Qwen3RMSNorm
model.layers.18: Qwen3DecoderLayer
model.layers.18.self_attn: Qwen3Attention
model.layers.18.self_attn.q_proj: Linear
model.layers.18.self_attn.k_proj: Linear
model.layers.18.self_attn.v_proj: Linear
model.layers.18.self_attn.o_proj: Linear
model.layers.18.self_attn.q_norm: Qwen3RMSNorm
model.layers.18.self_attn.k_norm: Qwen3RMSNorm
model.layers.18.mlp: Qwen3MLP
model.layers.18.mlp.gate_proj: Linear
model.layers.18.mlp.up_proj: Linear
model.layers.18.mlp.down_proj: Linear
model.layers.18.mlp.act_fn: SiLU
model.layers.18.input_layernorm: Qwen3RMSNorm
model.layers.18.post_attention_layernorm: Qwen3RMSNorm
model.layers.19: Qwen3DecoderLayer
model.layers.19.self_attn: Qwen3Attention
model.layers.19.self_attn.q_proj: Linear
model.layers.19.self_attn.k_proj: Linear
model.layers.19.self_attn.v_proj: Linear
model.layers.19.self_attn.o_proj: Linear
model.layers.19.self_attn.q_norm: Qwen3RMSNorm
model.layers.19.self_attn.k_norm: Qwen3RMSNorm
model.layers.19.mlp: Qwen3MLP
model.layers.19.mlp.gate_proj: Linear
model.layers.19.mlp.up_proj: Linear
model.layers.19.mlp.down_proj: Linear
model.layers.19.mlp.act_fn: SiLU
model.layers.19.input_layernorm: Qwen3RMSNorm
model.layers.19.post_attention_layernorm: Qwen3RMSNorm
model.layers.20: Qwen3DecoderLayer
model.layers.20.self_attn: Qwen3Attention
model.layers.20.self_attn.q_proj: Linear
model.layers.20.self_attn.k_proj: Linear
model.layers.20.self_attn.v_proj: Linear
model.layers.20.self_attn.o_proj: Linear
model.layers.20.self_attn.q_norm: Qwen3RMSNorm
model.layers.20.self_attn.k_norm: Qwen3RMSNorm
model.layers.20.mlp: Qwen3MLP
model.layers.20.mlp.gate_proj: Linear
model.layers.20.mlp.up_proj: Linear
model.layers.20.mlp.down_proj: Linear
model.layers.20.mlp.act_fn: SiLU
model.layers.20.input_layernorm: Qwen3RMSNorm
model.layers.20.post_attention_layernorm: Qwen3RMSNorm
model.layers.21: Qwen3DecoderLayer
model.layers.21.self_attn: Qwen3Attention
model.layers.21.self_attn.q_proj: Linear
model.layers.21.self_attn.k_proj: Linear
model.layers.21.self_attn.v_proj: Linear
model.layers.21.self_attn.o_proj: Linear
model.layers.21.self_attn.q_norm: Qwen3RMSNorm
model.layers.21.self_attn.k_norm: Qwen3RMSNorm
model.layers.21.mlp: Qwen3MLP
model.layers.21.mlp.gate_proj: Linear
model.layers.21.mlp.up_proj: Linear
model.layers.21.mlp.down_proj: Linear
model.layers.21.mlp.act_fn: SiLU
model.layers.21.input_layernorm: Qwen3RMSNorm
model.layers.21.post_attention_layernorm: Qwen3RMSNorm
model.layers.22: Qwen3DecoderLayer
model.layers.22.self_attn: Qwen3Attention
model.layers.22.self_attn.q_proj: Linear
model.layers.22.self_attn.k_proj: Linear
model.layers.22.self_attn.v_proj: Linear
model.layers.22.self_attn.o_proj: Linear
model.layers.22.self_attn.q_norm: Qwen3RMSNorm
model.layers.22.self_attn.k_norm: Qwen3RMSNorm
model.layers.22.mlp: Qwen3MLP
model.layers.22.mlp.gate_proj: Linear
model.layers.22.mlp.up_proj: Linear
model.layers.22.mlp.down_proj: Linear
model.layers.22.mlp.act_fn: SiLU
model.layers.22.input_layernorm: Qwen3RMSNorm
model.layers.22.post_attention_layernorm: Qwen3RMSNorm
model.layers.23: Qwen3DecoderLayer
model.layers.23.self_attn: Qwen3Attention
model.layers.23.self_attn.q_proj: Linear
model.layers.23.self_attn.k_proj: Linear
model.layers.23.self_attn.v_proj: Linear
model.layers.23.self_attn.o_proj: Linear
model.layers.23.self_attn.q_norm: Qwen3RMSNorm
model.layers.23.self_attn.k_norm: Qwen3RMSNorm
model.layers.23.mlp: Qwen3MLP
model.layers.23.mlp.gate_proj: Linear
model.layers.23.mlp.up_proj: Linear
model.layers.23.mlp.down_proj: Linear
model.layers.23.mlp.act_fn: SiLU
model.layers.23.input_layernorm: Qwen3RMSNorm
model.layers.23.post_attention_layernorm: Qwen3RMSNorm
model.layers.24: Qwen3DecoderLayer
model.layers.24.self_attn: Qwen3Attention
model.layers.24.self_attn.q_proj: Linear
model.layers.24.self_attn.k_proj: Linear
model.layers.24.self_attn.v_proj: Linear
model.layers.24.self_attn.o_proj: Linear
model.layers.24.self_attn.q_norm: Qwen3RMSNorm
model.layers.24.self_attn.k_norm: Qwen3RMSNorm
model.layers.24.mlp: Qwen3MLP
model.layers.24.mlp.gate_proj: Linear
model.layers.24.mlp.up_proj: Linear
model.layers.24.mlp.down_proj: Linear
model.layers.24.mlp.act_fn: SiLU
model.layers.24.input_layernorm: Qwen3RMSNorm
model.layers.24.post_attention_layernorm: Qwen3RMSNorm
model.layers.25: Qwen3DecoderLayer
model.layers.25.self_attn: Qwen3Attention
model.layers.25.self_attn.q_proj: Linear
model.layers.25.self_attn.k_proj: Linear
model.layers.25.self_attn.v_proj: Linear
model.layers.25.self_attn.o_proj: Linear
model.layers.25.self_attn.q_norm: Qwen3RMSNorm
model.layers.25.self_attn.k_norm: Qwen3RMSNorm
model.layers.25.mlp: Qwen3MLP
model.layers.25.mlp.gate_proj: Linear
model.layers.25.mlp.up_proj: Linear
model.layers.25.mlp.down_proj: Linear
model.layers.25.mlp.act_fn: SiLU
model.layers.25.input_layernorm: Qwen3RMSNorm
model.layers.25.post_attention_layernorm: Qwen3RMSNorm
model.layers.26: Qwen3DecoderLayer
model.layers.26.self_attn: Qwen3Attention
model.layers.26.self_attn.q_proj: Linear
model.layers.26.self_attn.k_proj: Linear
model.layers.26.self_attn.v_proj: Linear
model.layers.26.self_attn.o_proj: Linear
model.layers.26.self_attn.q_norm: Qwen3RMSNorm
model.layers.26.self_attn.k_norm: Qwen3RMSNorm
model.layers.26.mlp: Qwen3MLP
model.layers.26.mlp.gate_proj: Linear
model.layers.26.mlp.up_proj: Linear
model.layers.26.mlp.down_proj: Linear
model.layers.26.mlp.act_fn: SiLU
model.layers.26.input_layernorm: Qwen3RMSNorm
model.layers.26.post_attention_layernorm: Qwen3RMSNorm
model.layers.27: Qwen3DecoderLayer
model.layers.27.self_attn: Qwen3Attention
model.layers.27.self_attn.q_proj: Linear
model.layers.27.self_attn.k_proj: Linear
model.layers.27.self_attn.v_proj: Linear
model.layers.27.self_attn.o_proj: Linear
model.layers.27.self_attn.q_norm: Qwen3RMSNorm
model.layers.27.self_attn.k_norm: Qwen3RMSNorm
model.layers.27.mlp: Qwen3MLP
model.layers.27.mlp.gate_proj: Linear
model.layers.27.mlp.up_proj: Linear
model.layers.27.mlp.down_proj: Linear
model.layers.27.mlp.act_fn: SiLU
model.layers.27.input_layernorm: Qwen3RMSNorm
model.layers.27.post_attention_layernorm: Qwen3RMSNorm
model.norm: Qwen3RMSNorm
model.rotary_emb: Qwen3RotaryEmbedding
model.vision_tower: TATokVisionTower
model.vision_tower.vision_tower: TextAlignedTokenizer
model.vision_tower.vision_tower.encoder: SiglipVisionTransformer
model.vision_tower.vision_tower.encoder.embeddings: SiglipVisionEmbeddings
model.vision_tower.vision_tower.encoder.embeddings.patch_embedding: Conv2d
model.vision_tower.vision_tower.encoder.embeddings.position_embedding: Embedding
model.vision_tower.vision_tower.encoder.encoder: SiglipEncoder
model.vision_tower.vision_tower.encoder.encoder.layers: ModuleList
model.vision_tower.vision_tower.encoder.encoder.layers.0: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.0.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.0.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.0.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.0.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.0.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.0.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.0.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.0.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.0.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.0.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.0.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.1: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.1.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.1.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.1.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.1.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.1.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.1.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.1.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.1.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.1.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.1.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.1.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.2: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.2.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.2.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.2.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.2.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.2.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.2.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.2.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.2.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.2.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.2.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.2.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.3: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.3.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.3.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.3.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.3.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.3.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.3.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.3.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.3.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.3.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.3.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.3.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.4: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.4.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.4.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.4.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.4.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.4.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.4.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.4.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.4.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.4.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.4.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.4.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.5: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.5.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.5.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.5.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.5.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.5.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.5.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.5.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.5.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.5.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.5.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.5.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.6: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.6.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.6.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.6.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.6.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.6.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.6.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.6.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.6.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.6.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.6.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.6.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.7: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.7.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.7.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.7.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.7.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.7.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.7.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.7.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.7.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.7.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.7.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.7.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.8: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.8.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.8.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.8.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.8.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.8.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.8.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.8.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.8.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.8.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.8.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.8.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.9: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.9.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.9.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.9.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.9.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.9.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.9.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.9.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.9.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.9.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.9.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.9.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.10: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.10.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.10.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.10.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.10.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.10.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.10.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.10.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.10.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.10.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.10.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.10.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.11: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.11.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.11.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.11.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.11.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.11.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.11.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.11.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.11.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.11.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.11.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.11.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.12: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.12.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.12.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.12.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.12.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.12.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.12.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.12.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.12.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.12.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.12.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.12.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.13: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.13.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.13.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.13.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.13.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.13.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.13.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.13.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.13.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.13.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.13.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.13.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.14: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.14.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.14.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.14.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.14.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.14.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.14.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.14.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.14.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.14.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.14.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.14.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.15: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.15.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.15.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.15.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.15.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.15.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.15.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.15.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.15.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.15.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.15.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.15.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.16: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.16.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.16.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.16.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.16.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.16.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.16.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.16.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.16.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.16.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.16.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.16.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.17: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.17.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.17.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.17.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.17.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.17.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.17.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.17.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.17.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.17.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.17.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.17.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.18: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.18.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.18.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.18.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.18.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.18.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.18.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.18.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.18.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.18.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.18.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.18.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.19: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.19.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.19.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.19.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.19.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.19.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.19.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.19.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.19.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.19.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.19.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.19.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.20: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.20.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.20.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.20.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.20.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.20.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.20.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.20.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.20.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.20.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.20.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.20.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.21: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.21.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.21.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.21.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.21.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.21.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.21.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.21.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.21.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.21.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.21.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.21.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.22: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.22.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.22.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.22.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.22.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.22.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.22.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.22.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.22.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.22.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.22.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.22.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.23: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.23.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.23.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.23.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.23.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.23.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.23.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.23.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.23.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.23.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.23.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.23.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.24: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.24.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.24.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.24.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.24.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.24.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.24.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.24.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.24.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.24.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.24.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.24.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.25: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.25.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.25.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.25.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.25.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.25.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.25.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.25.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.25.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.25.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.25.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.25.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.26: SiglipEncoderLayer
model.vision_tower.vision_tower.encoder.encoder.layers.26.layer_norm1: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.26.self_attn: SiglipAttention
model.vision_tower.vision_tower.encoder.encoder.layers.26.self_attn.k_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.26.self_attn.v_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.26.self_attn.q_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.26.self_attn.out_proj: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.26.layer_norm2: LayerNorm
model.vision_tower.vision_tower.encoder.encoder.layers.26.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.encoder.layers.26.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.encoder.layers.26.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.encoder.layers.26.mlp.fc2: Linear
model.vision_tower.vision_tower.encoder.post_layernorm: LayerNorm
model.vision_tower.vision_tower.encoder.head: SiglipMultiheadAttentionPoolingHead
model.vision_tower.vision_tower.encoder.head.attention: MultiheadAttention
model.vision_tower.vision_tower.encoder.head.attention.out_proj: NonDynamicallyQuantizableLinear
model.vision_tower.vision_tower.encoder.head.layernorm: LayerNorm
model.vision_tower.vision_tower.encoder.head.mlp: SiglipMLP
model.vision_tower.vision_tower.encoder.head.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.encoder.head.mlp.fc1: Linear
model.vision_tower.vision_tower.encoder.head.mlp.fc2: Linear
model.vision_tower.vision_tower.decoder: Siglip2VisionModel
model.vision_tower.vision_tower.decoder.vision_model: Siglip2VisionTransformer
model.vision_tower.vision_tower.decoder.vision_model.embeddings: Siglip2VisionEmbeddings
model.vision_tower.vision_tower.decoder.vision_model.embeddings.patch_embedding: Linear
model.vision_tower.vision_tower.decoder.vision_model.embeddings.position_embedding: Embedding
model.vision_tower.vision_tower.decoder.vision_model.encoder: Siglip2Encoder
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers: ModuleList
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0: Siglip2EncoderLayer
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.layer_norm1: LayerNorm
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.self_attn: Siglip2Attention
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.self_attn.k_proj: Linear
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.self_attn.v_proj: Linear
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.self_attn.q_proj: Linear
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.self_attn.out_proj: Linear
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.layer_norm2: LayerNorm
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.mlp: Siglip2MLP
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.mlp.fc1: Linear
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.mlp.fc2: Linear
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1: Siglip2EncoderLayer
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.layer_norm1: LayerNorm
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.self_attn: Siglip2Attention
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.self_attn.k_proj: Linear
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.self_attn.v_proj: Linear
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.self_attn.q_proj: Linear
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.self_attn.out_proj: Linear
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.layer_norm2: LayerNorm
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.mlp: Siglip2MLP
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.mlp.fc1: Linear
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.mlp.fc2: Linear
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2: Siglip2EncoderLayer
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.layer_norm1: LayerNorm
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.self_attn: Siglip2Attention
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.self_attn.k_proj: Linear
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.self_attn.v_proj: Linear
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.self_attn.q_proj: Linear
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.self_attn.out_proj: Linear
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.layer_norm2: LayerNorm
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.mlp: Siglip2MLP
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.mlp.fc1: Linear
model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.mlp.fc2: Linear
model.vision_tower.vision_tower.decoder.vision_model.post_layernorm: LayerNorm
model.vision_tower.vision_tower.decoder.vision_model.head: Siglip2MultiheadAttentionPoolingHead
model.vision_tower.vision_tower.decoder.vision_model.head.attention: MultiheadAttention
model.vision_tower.vision_tower.decoder.vision_model.head.attention.out_proj: NonDynamicallyQuantizableLinear
model.vision_tower.vision_tower.decoder.vision_model.head.layernorm: LayerNorm
model.vision_tower.vision_tower.decoder.vision_model.head.mlp: Siglip2MLP
model.vision_tower.vision_tower.decoder.vision_model.head.mlp.activation_fn: PytorchGELUTanh
model.vision_tower.vision_tower.decoder.vision_model.head.mlp.fc1: Linear
model.vision_tower.vision_tower.decoder.vision_model.head.mlp.fc2: Linear
model.vision_tower.vision_tower.encode_task_layer: Sequential
model.vision_tower.vision_tower.encode_task_layer.0: Linear
model.vision_tower.vision_tower.encode_task_layer.1: Tanh
model.vision_tower.vision_tower.decode_task_layer: Sequential
model.vision_tower.vision_tower.decode_task_layer.0: Linear
model.vision_tower.vision_tower.decode_task_layer.1: Tanh
model.vision_tower.vision_tower.decode_task_layer.2: Linear
model.vision_tower.vision_tower.bottleneck: Bottleneck
model.vision_tower.vision_tower.bottleneck.in_linear: Linear
model.vision_tower.vision_tower.bottleneck.out_linear: Linear
model.vision_tower.vision_tower.bottleneck.regularizer: SimVectorQuantizer
model.vision_tower.vision_tower.bottleneck.regularizer.embedding: Embedding
model.vision_tower.vision_tower.bottleneck.regularizer.embedding_proj: Linear
model.vision_tower.vision_tower.scale_layer: ScalingLayer
model.vision_tower.vision_tower.image_resize: Resize
model.sana: SanaTransformer2DModel
model.sana.patch_embed: PatchEmbed
model.sana.patch_embed.proj: Conv2d
model.sana.time_embed: AdaLayerNormSingle
model.sana.time_embed.emb: PixArtAlphaCombinedTimestepSizeEmbeddings
model.sana.time_embed.emb.time_proj: Timesteps
model.sana.time_embed.emb.timestep_embedder: TimestepEmbedding
model.sana.time_embed.emb.timestep_embedder.linear_1: Linear
model.sana.time_embed.emb.timestep_embedder.act: SiLU
model.sana.time_embed.emb.timestep_embedder.linear_2: Linear
model.sana.time_embed.silu: SiLU
model.sana.time_embed.linear: Linear
model.sana.caption_projection: PixArtAlphaTextProjection
model.sana.caption_projection.linear_1: Linear
model.sana.caption_projection.act_1: GELU
model.sana.caption_projection.linear_2: Linear
model.sana.caption_norm: RMSNorm
model.sana.transformer_blocks: ModuleList
model.sana.transformer_blocks.0: SanaTransformerBlock
model.sana.transformer_blocks.0.norm1: LayerNorm
model.sana.transformer_blocks.0.attn1: Attention
model.sana.transformer_blocks.0.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.0.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.0.attn1.to_q: Linear
model.sana.transformer_blocks.0.attn1.to_k: Linear
model.sana.transformer_blocks.0.attn1.to_v: Linear
model.sana.transformer_blocks.0.attn1.to_out: ModuleList
model.sana.transformer_blocks.0.attn1.to_out.0: Linear
model.sana.transformer_blocks.0.attn1.to_out.1: Dropout
model.sana.transformer_blocks.0.norm2: LayerNorm
model.sana.transformer_blocks.0.attn2: Attention
model.sana.transformer_blocks.0.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.0.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.0.attn2.to_q: Linear
model.sana.transformer_blocks.0.attn2.to_k: Linear
model.sana.transformer_blocks.0.attn2.to_v: Linear
model.sana.transformer_blocks.0.attn2.to_out: ModuleList
model.sana.transformer_blocks.0.attn2.to_out.0: Linear
model.sana.transformer_blocks.0.attn2.to_out.1: Dropout
model.sana.transformer_blocks.0.ff: GLUMBConv
model.sana.transformer_blocks.0.ff.nonlinearity: SiLU
model.sana.transformer_blocks.0.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.0.ff.conv_depth: Conv2d
model.sana.transformer_blocks.0.ff.conv_point: Conv2d
model.sana.transformer_blocks.1: SanaTransformerBlock
model.sana.transformer_blocks.1.norm1: LayerNorm
model.sana.transformer_blocks.1.attn1: Attention
model.sana.transformer_blocks.1.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.1.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.1.attn1.to_q: Linear
model.sana.transformer_blocks.1.attn1.to_k: Linear
model.sana.transformer_blocks.1.attn1.to_v: Linear
model.sana.transformer_blocks.1.attn1.to_out: ModuleList
model.sana.transformer_blocks.1.attn1.to_out.0: Linear
model.sana.transformer_blocks.1.attn1.to_out.1: Dropout
model.sana.transformer_blocks.1.norm2: LayerNorm
model.sana.transformer_blocks.1.attn2: Attention
model.sana.transformer_blocks.1.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.1.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.1.attn2.to_q: Linear
model.sana.transformer_blocks.1.attn2.to_k: Linear
model.sana.transformer_blocks.1.attn2.to_v: Linear
model.sana.transformer_blocks.1.attn2.to_out: ModuleList
model.sana.transformer_blocks.1.attn2.to_out.0: Linear
model.sana.transformer_blocks.1.attn2.to_out.1: Dropout
model.sana.transformer_blocks.1.ff: GLUMBConv
model.sana.transformer_blocks.1.ff.nonlinearity: SiLU
model.sana.transformer_blocks.1.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.1.ff.conv_depth: Conv2d
model.sana.transformer_blocks.1.ff.conv_point: Conv2d
model.sana.transformer_blocks.2: SanaTransformerBlock
model.sana.transformer_blocks.2.norm1: LayerNorm
model.sana.transformer_blocks.2.attn1: Attention
model.sana.transformer_blocks.2.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.2.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.2.attn1.to_q: Linear
model.sana.transformer_blocks.2.attn1.to_k: Linear
model.sana.transformer_blocks.2.attn1.to_v: Linear
model.sana.transformer_blocks.2.attn1.to_out: ModuleList
model.sana.transformer_blocks.2.attn1.to_out.0: Linear
model.sana.transformer_blocks.2.attn1.to_out.1: Dropout
model.sana.transformer_blocks.2.norm2: LayerNorm
model.sana.transformer_blocks.2.attn2: Attention
model.sana.transformer_blocks.2.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.2.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.2.attn2.to_q: Linear
model.sana.transformer_blocks.2.attn2.to_k: Linear
model.sana.transformer_blocks.2.attn2.to_v: Linear
model.sana.transformer_blocks.2.attn2.to_out: ModuleList
model.sana.transformer_blocks.2.attn2.to_out.0: Linear
model.sana.transformer_blocks.2.attn2.to_out.1: Dropout
model.sana.transformer_blocks.2.ff: GLUMBConv
model.sana.transformer_blocks.2.ff.nonlinearity: SiLU
model.sana.transformer_blocks.2.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.2.ff.conv_depth: Conv2d
model.sana.transformer_blocks.2.ff.conv_point: Conv2d
model.sana.transformer_blocks.3: SanaTransformerBlock
model.sana.transformer_blocks.3.norm1: LayerNorm
model.sana.transformer_blocks.3.attn1: Attention
model.sana.transformer_blocks.3.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.3.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.3.attn1.to_q: Linear
model.sana.transformer_blocks.3.attn1.to_k: Linear
model.sana.transformer_blocks.3.attn1.to_v: Linear
model.sana.transformer_blocks.3.attn1.to_out: ModuleList
model.sana.transformer_blocks.3.attn1.to_out.0: Linear
model.sana.transformer_blocks.3.attn1.to_out.1: Dropout
model.sana.transformer_blocks.3.norm2: LayerNorm
model.sana.transformer_blocks.3.attn2: Attention
model.sana.transformer_blocks.3.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.3.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.3.attn2.to_q: Linear
model.sana.transformer_blocks.3.attn2.to_k: Linear
model.sana.transformer_blocks.3.attn2.to_v: Linear
model.sana.transformer_blocks.3.attn2.to_out: ModuleList
model.sana.transformer_blocks.3.attn2.to_out.0: Linear
model.sana.transformer_blocks.3.attn2.to_out.1: Dropout
model.sana.transformer_blocks.3.ff: GLUMBConv
model.sana.transformer_blocks.3.ff.nonlinearity: SiLU
model.sana.transformer_blocks.3.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.3.ff.conv_depth: Conv2d
model.sana.transformer_blocks.3.ff.conv_point: Conv2d
model.sana.transformer_blocks.4: SanaTransformerBlock
model.sana.transformer_blocks.4.norm1: LayerNorm
model.sana.transformer_blocks.4.attn1: Attention
model.sana.transformer_blocks.4.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.4.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.4.attn1.to_q: Linear
model.sana.transformer_blocks.4.attn1.to_k: Linear
model.sana.transformer_blocks.4.attn1.to_v: Linear
model.sana.transformer_blocks.4.attn1.to_out: ModuleList
model.sana.transformer_blocks.4.attn1.to_out.0: Linear
model.sana.transformer_blocks.4.attn1.to_out.1: Dropout
model.sana.transformer_blocks.4.norm2: LayerNorm
model.sana.transformer_blocks.4.attn2: Attention
model.sana.transformer_blocks.4.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.4.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.4.attn2.to_q: Linear
model.sana.transformer_blocks.4.attn2.to_k: Linear
model.sana.transformer_blocks.4.attn2.to_v: Linear
model.sana.transformer_blocks.4.attn2.to_out: ModuleList
model.sana.transformer_blocks.4.attn2.to_out.0: Linear
model.sana.transformer_blocks.4.attn2.to_out.1: Dropout
model.sana.transformer_blocks.4.ff: GLUMBConv
model.sana.transformer_blocks.4.ff.nonlinearity: SiLU
model.sana.transformer_blocks.4.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.4.ff.conv_depth: Conv2d
model.sana.transformer_blocks.4.ff.conv_point: Conv2d
model.sana.transformer_blocks.5: SanaTransformerBlock
model.sana.transformer_blocks.5.norm1: LayerNorm
model.sana.transformer_blocks.5.attn1: Attention
model.sana.transformer_blocks.5.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.5.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.5.attn1.to_q: Linear
model.sana.transformer_blocks.5.attn1.to_k: Linear
model.sana.transformer_blocks.5.attn1.to_v: Linear
model.sana.transformer_blocks.5.attn1.to_out: ModuleList
model.sana.transformer_blocks.5.attn1.to_out.0: Linear
model.sana.transformer_blocks.5.attn1.to_out.1: Dropout
model.sana.transformer_blocks.5.norm2: LayerNorm
model.sana.transformer_blocks.5.attn2: Attention
model.sana.transformer_blocks.5.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.5.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.5.attn2.to_q: Linear
model.sana.transformer_blocks.5.attn2.to_k: Linear
model.sana.transformer_blocks.5.attn2.to_v: Linear
model.sana.transformer_blocks.5.attn2.to_out: ModuleList
model.sana.transformer_blocks.5.attn2.to_out.0: Linear
model.sana.transformer_blocks.5.attn2.to_out.1: Dropout
model.sana.transformer_blocks.5.ff: GLUMBConv
model.sana.transformer_blocks.5.ff.nonlinearity: SiLU
model.sana.transformer_blocks.5.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.5.ff.conv_depth: Conv2d
model.sana.transformer_blocks.5.ff.conv_point: Conv2d
model.sana.transformer_blocks.6: SanaTransformerBlock
model.sana.transformer_blocks.6.norm1: LayerNorm
model.sana.transformer_blocks.6.attn1: Attention
model.sana.transformer_blocks.6.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.6.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.6.attn1.to_q: Linear
model.sana.transformer_blocks.6.attn1.to_k: Linear
model.sana.transformer_blocks.6.attn1.to_v: Linear
model.sana.transformer_blocks.6.attn1.to_out: ModuleList
model.sana.transformer_blocks.6.attn1.to_out.0: Linear
model.sana.transformer_blocks.6.attn1.to_out.1: Dropout
model.sana.transformer_blocks.6.norm2: LayerNorm
model.sana.transformer_blocks.6.attn2: Attention
model.sana.transformer_blocks.6.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.6.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.6.attn2.to_q: Linear
model.sana.transformer_blocks.6.attn2.to_k: Linear
model.sana.transformer_blocks.6.attn2.to_v: Linear
model.sana.transformer_blocks.6.attn2.to_out: ModuleList
model.sana.transformer_blocks.6.attn2.to_out.0: Linear
model.sana.transformer_blocks.6.attn2.to_out.1: Dropout
model.sana.transformer_blocks.6.ff: GLUMBConv
model.sana.transformer_blocks.6.ff.nonlinearity: SiLU
model.sana.transformer_blocks.6.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.6.ff.conv_depth: Conv2d
model.sana.transformer_blocks.6.ff.conv_point: Conv2d
model.sana.transformer_blocks.7: SanaTransformerBlock
model.sana.transformer_blocks.7.norm1: LayerNorm
model.sana.transformer_blocks.7.attn1: Attention
model.sana.transformer_blocks.7.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.7.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.7.attn1.to_q: Linear
model.sana.transformer_blocks.7.attn1.to_k: Linear
model.sana.transformer_blocks.7.attn1.to_v: Linear
model.sana.transformer_blocks.7.attn1.to_out: ModuleList
model.sana.transformer_blocks.7.attn1.to_out.0: Linear
model.sana.transformer_blocks.7.attn1.to_out.1: Dropout
model.sana.transformer_blocks.7.norm2: LayerNorm
model.sana.transformer_blocks.7.attn2: Attention
model.sana.transformer_blocks.7.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.7.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.7.attn2.to_q: Linear
model.sana.transformer_blocks.7.attn2.to_k: Linear
model.sana.transformer_blocks.7.attn2.to_v: Linear
model.sana.transformer_blocks.7.attn2.to_out: ModuleList
model.sana.transformer_blocks.7.attn2.to_out.0: Linear
model.sana.transformer_blocks.7.attn2.to_out.1: Dropout
model.sana.transformer_blocks.7.ff: GLUMBConv
model.sana.transformer_blocks.7.ff.nonlinearity: SiLU
model.sana.transformer_blocks.7.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.7.ff.conv_depth: Conv2d
model.sana.transformer_blocks.7.ff.conv_point: Conv2d
model.sana.transformer_blocks.8: SanaTransformerBlock
model.sana.transformer_blocks.8.norm1: LayerNorm
model.sana.transformer_blocks.8.attn1: Attention
model.sana.transformer_blocks.8.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.8.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.8.attn1.to_q: Linear
model.sana.transformer_blocks.8.attn1.to_k: Linear
model.sana.transformer_blocks.8.attn1.to_v: Linear
model.sana.transformer_blocks.8.attn1.to_out: ModuleList
model.sana.transformer_blocks.8.attn1.to_out.0: Linear
model.sana.transformer_blocks.8.attn1.to_out.1: Dropout
model.sana.transformer_blocks.8.norm2: LayerNorm
model.sana.transformer_blocks.8.attn2: Attention
model.sana.transformer_blocks.8.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.8.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.8.attn2.to_q: Linear
model.sana.transformer_blocks.8.attn2.to_k: Linear
model.sana.transformer_blocks.8.attn2.to_v: Linear
model.sana.transformer_blocks.8.attn2.to_out: ModuleList
model.sana.transformer_blocks.8.attn2.to_out.0: Linear
model.sana.transformer_blocks.8.attn2.to_out.1: Dropout
model.sana.transformer_blocks.8.ff: GLUMBConv
model.sana.transformer_blocks.8.ff.nonlinearity: SiLU
model.sana.transformer_blocks.8.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.8.ff.conv_depth: Conv2d
model.sana.transformer_blocks.8.ff.conv_point: Conv2d
model.sana.transformer_blocks.9: SanaTransformerBlock
model.sana.transformer_blocks.9.norm1: LayerNorm
model.sana.transformer_blocks.9.attn1: Attention
model.sana.transformer_blocks.9.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.9.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.9.attn1.to_q: Linear
model.sana.transformer_blocks.9.attn1.to_k: Linear
model.sana.transformer_blocks.9.attn1.to_v: Linear
model.sana.transformer_blocks.9.attn1.to_out: ModuleList
model.sana.transformer_blocks.9.attn1.to_out.0: Linear
model.sana.transformer_blocks.9.attn1.to_out.1: Dropout
model.sana.transformer_blocks.9.norm2: LayerNorm
model.sana.transformer_blocks.9.attn2: Attention
model.sana.transformer_blocks.9.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.9.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.9.attn2.to_q: Linear
model.sana.transformer_blocks.9.attn2.to_k: Linear
model.sana.transformer_blocks.9.attn2.to_v: Linear
model.sana.transformer_blocks.9.attn2.to_out: ModuleList
model.sana.transformer_blocks.9.attn2.to_out.0: Linear
model.sana.transformer_blocks.9.attn2.to_out.1: Dropout
model.sana.transformer_blocks.9.ff: GLUMBConv
model.sana.transformer_blocks.9.ff.nonlinearity: SiLU
model.sana.transformer_blocks.9.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.9.ff.conv_depth: Conv2d
model.sana.transformer_blocks.9.ff.conv_point: Conv2d
model.sana.transformer_blocks.10: SanaTransformerBlock
model.sana.transformer_blocks.10.norm1: LayerNorm
model.sana.transformer_blocks.10.attn1: Attention
model.sana.transformer_blocks.10.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.10.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.10.attn1.to_q: Linear
model.sana.transformer_blocks.10.attn1.to_k: Linear
model.sana.transformer_blocks.10.attn1.to_v: Linear
model.sana.transformer_blocks.10.attn1.to_out: ModuleList
model.sana.transformer_blocks.10.attn1.to_out.0: Linear
model.sana.transformer_blocks.10.attn1.to_out.1: Dropout
model.sana.transformer_blocks.10.norm2: LayerNorm
model.sana.transformer_blocks.10.attn2: Attention
model.sana.transformer_blocks.10.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.10.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.10.attn2.to_q: Linear
model.sana.transformer_blocks.10.attn2.to_k: Linear
model.sana.transformer_blocks.10.attn2.to_v: Linear
model.sana.transformer_blocks.10.attn2.to_out: ModuleList
model.sana.transformer_blocks.10.attn2.to_out.0: Linear
model.sana.transformer_blocks.10.attn2.to_out.1: Dropout
model.sana.transformer_blocks.10.ff: GLUMBConv
model.sana.transformer_blocks.10.ff.nonlinearity: SiLU
model.sana.transformer_blocks.10.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.10.ff.conv_depth: Conv2d
model.sana.transformer_blocks.10.ff.conv_point: Conv2d
model.sana.transformer_blocks.11: SanaTransformerBlock
model.sana.transformer_blocks.11.norm1: LayerNorm
model.sana.transformer_blocks.11.attn1: Attention
model.sana.transformer_blocks.11.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.11.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.11.attn1.to_q: Linear
model.sana.transformer_blocks.11.attn1.to_k: Linear
model.sana.transformer_blocks.11.attn1.to_v: Linear
model.sana.transformer_blocks.11.attn1.to_out: ModuleList
model.sana.transformer_blocks.11.attn1.to_out.0: Linear
model.sana.transformer_blocks.11.attn1.to_out.1: Dropout
model.sana.transformer_blocks.11.norm2: LayerNorm
model.sana.transformer_blocks.11.attn2: Attention
model.sana.transformer_blocks.11.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.11.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.11.attn2.to_q: Linear
model.sana.transformer_blocks.11.attn2.to_k: Linear
model.sana.transformer_blocks.11.attn2.to_v: Linear
model.sana.transformer_blocks.11.attn2.to_out: ModuleList
model.sana.transformer_blocks.11.attn2.to_out.0: Linear
model.sana.transformer_blocks.11.attn2.to_out.1: Dropout
model.sana.transformer_blocks.11.ff: GLUMBConv
model.sana.transformer_blocks.11.ff.nonlinearity: SiLU
model.sana.transformer_blocks.11.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.11.ff.conv_depth: Conv2d
model.sana.transformer_blocks.11.ff.conv_point: Conv2d
model.sana.transformer_blocks.12: SanaTransformerBlock
model.sana.transformer_blocks.12.norm1: LayerNorm
model.sana.transformer_blocks.12.attn1: Attention
model.sana.transformer_blocks.12.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.12.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.12.attn1.to_q: Linear
model.sana.transformer_blocks.12.attn1.to_k: Linear
model.sana.transformer_blocks.12.attn1.to_v: Linear
model.sana.transformer_blocks.12.attn1.to_out: ModuleList
model.sana.transformer_blocks.12.attn1.to_out.0: Linear
model.sana.transformer_blocks.12.attn1.to_out.1: Dropout
model.sana.transformer_blocks.12.norm2: LayerNorm
model.sana.transformer_blocks.12.attn2: Attention
model.sana.transformer_blocks.12.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.12.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.12.attn2.to_q: Linear
model.sana.transformer_blocks.12.attn2.to_k: Linear
model.sana.transformer_blocks.12.attn2.to_v: Linear
model.sana.transformer_blocks.12.attn2.to_out: ModuleList
model.sana.transformer_blocks.12.attn2.to_out.0: Linear
model.sana.transformer_blocks.12.attn2.to_out.1: Dropout
model.sana.transformer_blocks.12.ff: GLUMBConv
model.sana.transformer_blocks.12.ff.nonlinearity: SiLU
model.sana.transformer_blocks.12.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.12.ff.conv_depth: Conv2d
model.sana.transformer_blocks.12.ff.conv_point: Conv2d
model.sana.transformer_blocks.13: SanaTransformerBlock
model.sana.transformer_blocks.13.norm1: LayerNorm
model.sana.transformer_blocks.13.attn1: Attention
model.sana.transformer_blocks.13.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.13.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.13.attn1.to_q: Linear
model.sana.transformer_blocks.13.attn1.to_k: Linear
model.sana.transformer_blocks.13.attn1.to_v: Linear
model.sana.transformer_blocks.13.attn1.to_out: ModuleList
model.sana.transformer_blocks.13.attn1.to_out.0: Linear
model.sana.transformer_blocks.13.attn1.to_out.1: Dropout
model.sana.transformer_blocks.13.norm2: LayerNorm
model.sana.transformer_blocks.13.attn2: Attention
model.sana.transformer_blocks.13.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.13.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.13.attn2.to_q: Linear
model.sana.transformer_blocks.13.attn2.to_k: Linear
model.sana.transformer_blocks.13.attn2.to_v: Linear
model.sana.transformer_blocks.13.attn2.to_out: ModuleList
model.sana.transformer_blocks.13.attn2.to_out.0: Linear
model.sana.transformer_blocks.13.attn2.to_out.1: Dropout
model.sana.transformer_blocks.13.ff: GLUMBConv
model.sana.transformer_blocks.13.ff.nonlinearity: SiLU
model.sana.transformer_blocks.13.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.13.ff.conv_depth: Conv2d
model.sana.transformer_blocks.13.ff.conv_point: Conv2d
model.sana.transformer_blocks.14: SanaTransformerBlock
model.sana.transformer_blocks.14.norm1: LayerNorm
model.sana.transformer_blocks.14.attn1: Attention
model.sana.transformer_blocks.14.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.14.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.14.attn1.to_q: Linear
model.sana.transformer_blocks.14.attn1.to_k: Linear
model.sana.transformer_blocks.14.attn1.to_v: Linear
model.sana.transformer_blocks.14.attn1.to_out: ModuleList
model.sana.transformer_blocks.14.attn1.to_out.0: Linear
model.sana.transformer_blocks.14.attn1.to_out.1: Dropout
model.sana.transformer_blocks.14.norm2: LayerNorm
model.sana.transformer_blocks.14.attn2: Attention
model.sana.transformer_blocks.14.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.14.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.14.attn2.to_q: Linear
model.sana.transformer_blocks.14.attn2.to_k: Linear
model.sana.transformer_blocks.14.attn2.to_v: Linear
model.sana.transformer_blocks.14.attn2.to_out: ModuleList
model.sana.transformer_blocks.14.attn2.to_out.0: Linear
model.sana.transformer_blocks.14.attn2.to_out.1: Dropout
model.sana.transformer_blocks.14.ff: GLUMBConv
model.sana.transformer_blocks.14.ff.nonlinearity: SiLU
model.sana.transformer_blocks.14.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.14.ff.conv_depth: Conv2d
model.sana.transformer_blocks.14.ff.conv_point: Conv2d
model.sana.transformer_blocks.15: SanaTransformerBlock
model.sana.transformer_blocks.15.norm1: LayerNorm
model.sana.transformer_blocks.15.attn1: Attention
model.sana.transformer_blocks.15.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.15.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.15.attn1.to_q: Linear
model.sana.transformer_blocks.15.attn1.to_k: Linear
model.sana.transformer_blocks.15.attn1.to_v: Linear
model.sana.transformer_blocks.15.attn1.to_out: ModuleList
model.sana.transformer_blocks.15.attn1.to_out.0: Linear
model.sana.transformer_blocks.15.attn1.to_out.1: Dropout
model.sana.transformer_blocks.15.norm2: LayerNorm
model.sana.transformer_blocks.15.attn2: Attention
model.sana.transformer_blocks.15.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.15.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.15.attn2.to_q: Linear
model.sana.transformer_blocks.15.attn2.to_k: Linear
model.sana.transformer_blocks.15.attn2.to_v: Linear
model.sana.transformer_blocks.15.attn2.to_out: ModuleList
model.sana.transformer_blocks.15.attn2.to_out.0: Linear
model.sana.transformer_blocks.15.attn2.to_out.1: Dropout
model.sana.transformer_blocks.15.ff: GLUMBConv
model.sana.transformer_blocks.15.ff.nonlinearity: SiLU
model.sana.transformer_blocks.15.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.15.ff.conv_depth: Conv2d
model.sana.transformer_blocks.15.ff.conv_point: Conv2d
model.sana.transformer_blocks.16: SanaTransformerBlock
model.sana.transformer_blocks.16.norm1: LayerNorm
model.sana.transformer_blocks.16.attn1: Attention
model.sana.transformer_blocks.16.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.16.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.16.attn1.to_q: Linear
model.sana.transformer_blocks.16.attn1.to_k: Linear
model.sana.transformer_blocks.16.attn1.to_v: Linear
model.sana.transformer_blocks.16.attn1.to_out: ModuleList
model.sana.transformer_blocks.16.attn1.to_out.0: Linear
model.sana.transformer_blocks.16.attn1.to_out.1: Dropout
model.sana.transformer_blocks.16.norm2: LayerNorm
model.sana.transformer_blocks.16.attn2: Attention
model.sana.transformer_blocks.16.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.16.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.16.attn2.to_q: Linear
model.sana.transformer_blocks.16.attn2.to_k: Linear
model.sana.transformer_blocks.16.attn2.to_v: Linear
model.sana.transformer_blocks.16.attn2.to_out: ModuleList
model.sana.transformer_blocks.16.attn2.to_out.0: Linear
model.sana.transformer_blocks.16.attn2.to_out.1: Dropout
model.sana.transformer_blocks.16.ff: GLUMBConv
model.sana.transformer_blocks.16.ff.nonlinearity: SiLU
model.sana.transformer_blocks.16.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.16.ff.conv_depth: Conv2d
model.sana.transformer_blocks.16.ff.conv_point: Conv2d
model.sana.transformer_blocks.17: SanaTransformerBlock
model.sana.transformer_blocks.17.norm1: LayerNorm
model.sana.transformer_blocks.17.attn1: Attention
model.sana.transformer_blocks.17.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.17.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.17.attn1.to_q: Linear
model.sana.transformer_blocks.17.attn1.to_k: Linear
model.sana.transformer_blocks.17.attn1.to_v: Linear
model.sana.transformer_blocks.17.attn1.to_out: ModuleList
model.sana.transformer_blocks.17.attn1.to_out.0: Linear
model.sana.transformer_blocks.17.attn1.to_out.1: Dropout
model.sana.transformer_blocks.17.norm2: LayerNorm
model.sana.transformer_blocks.17.attn2: Attention
model.sana.transformer_blocks.17.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.17.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.17.attn2.to_q: Linear
model.sana.transformer_blocks.17.attn2.to_k: Linear
model.sana.transformer_blocks.17.attn2.to_v: Linear
model.sana.transformer_blocks.17.attn2.to_out: ModuleList
model.sana.transformer_blocks.17.attn2.to_out.0: Linear
model.sana.transformer_blocks.17.attn2.to_out.1: Dropout
model.sana.transformer_blocks.17.ff: GLUMBConv
model.sana.transformer_blocks.17.ff.nonlinearity: SiLU
model.sana.transformer_blocks.17.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.17.ff.conv_depth: Conv2d
model.sana.transformer_blocks.17.ff.conv_point: Conv2d
model.sana.transformer_blocks.18: SanaTransformerBlock
model.sana.transformer_blocks.18.norm1: LayerNorm
model.sana.transformer_blocks.18.attn1: Attention
model.sana.transformer_blocks.18.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.18.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.18.attn1.to_q: Linear
model.sana.transformer_blocks.18.attn1.to_k: Linear
model.sana.transformer_blocks.18.attn1.to_v: Linear
model.sana.transformer_blocks.18.attn1.to_out: ModuleList
model.sana.transformer_blocks.18.attn1.to_out.0: Linear
model.sana.transformer_blocks.18.attn1.to_out.1: Dropout
model.sana.transformer_blocks.18.norm2: LayerNorm
model.sana.transformer_blocks.18.attn2: Attention
model.sana.transformer_blocks.18.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.18.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.18.attn2.to_q: Linear
model.sana.transformer_blocks.18.attn2.to_k: Linear
model.sana.transformer_blocks.18.attn2.to_v: Linear
model.sana.transformer_blocks.18.attn2.to_out: ModuleList
model.sana.transformer_blocks.18.attn2.to_out.0: Linear
model.sana.transformer_blocks.18.attn2.to_out.1: Dropout
model.sana.transformer_blocks.18.ff: GLUMBConv
model.sana.transformer_blocks.18.ff.nonlinearity: SiLU
model.sana.transformer_blocks.18.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.18.ff.conv_depth: Conv2d
model.sana.transformer_blocks.18.ff.conv_point: Conv2d
model.sana.transformer_blocks.19: SanaTransformerBlock
model.sana.transformer_blocks.19.norm1: LayerNorm
model.sana.transformer_blocks.19.attn1: Attention
model.sana.transformer_blocks.19.attn1.norm_q: RMSNorm
model.sana.transformer_blocks.19.attn1.norm_k: RMSNorm
model.sana.transformer_blocks.19.attn1.to_q: Linear
model.sana.transformer_blocks.19.attn1.to_k: Linear
model.sana.transformer_blocks.19.attn1.to_v: Linear
model.sana.transformer_blocks.19.attn1.to_out: ModuleList
model.sana.transformer_blocks.19.attn1.to_out.0: Linear
model.sana.transformer_blocks.19.attn1.to_out.1: Dropout
model.sana.transformer_blocks.19.norm2: LayerNorm
model.sana.transformer_blocks.19.attn2: Attention
model.sana.transformer_blocks.19.attn2.norm_q: RMSNorm
model.sana.transformer_blocks.19.attn2.norm_k: RMSNorm
model.sana.transformer_blocks.19.attn2.to_q: Linear
model.sana.transformer_blocks.19.attn2.to_k: Linear
model.sana.transformer_blocks.19.attn2.to_v: Linear
model.sana.transformer_blocks.19.attn2.to_out: ModuleList
model.sana.transformer_blocks.19.attn2.to_out.0: Linear
model.sana.transformer_blocks.19.attn2.to_out.1: Dropout
model.sana.transformer_blocks.19.ff: GLUMBConv
model.sana.transformer_blocks.19.ff.nonlinearity: SiLU
model.sana.transformer_blocks.19.ff.conv_inverted: Conv2d
model.sana.transformer_blocks.19.ff.conv_depth: Conv2d
model.sana.transformer_blocks.19.ff.conv_point: Conv2d
model.sana.norm_out: SanaModulatedNorm
model.sana.norm_out.norm: LayerNorm
model.sana.proj_out: Linear
model.sana_vae: AutoencoderDC
model.sana_vae.encoder: Encoder
model.sana_vae.encoder.conv_in: Conv2d
model.sana_vae.encoder.down_blocks: ModuleList
model.sana_vae.encoder.down_blocks.0: Sequential
model.sana_vae.encoder.down_blocks.0.0: ResBlock
model.sana_vae.encoder.down_blocks.0.0.nonlinearity: SiLU
model.sana_vae.encoder.down_blocks.0.0.conv1: Conv2d
model.sana_vae.encoder.down_blocks.0.0.conv2: Conv2d
model.sana_vae.encoder.down_blocks.0.0.norm: RMSNorm
model.sana_vae.encoder.down_blocks.0.1: ResBlock
model.sana_vae.encoder.down_blocks.0.1.nonlinearity: SiLU
model.sana_vae.encoder.down_blocks.0.1.conv1: Conv2d
model.sana_vae.encoder.down_blocks.0.1.conv2: Conv2d
model.sana_vae.encoder.down_blocks.0.1.norm: RMSNorm
model.sana_vae.encoder.down_blocks.0.2: DCDownBlock2d
model.sana_vae.encoder.down_blocks.0.2.conv: Conv2d
model.sana_vae.encoder.down_blocks.1: Sequential
model.sana_vae.encoder.down_blocks.1.0: ResBlock
model.sana_vae.encoder.down_blocks.1.0.nonlinearity: SiLU
model.sana_vae.encoder.down_blocks.1.0.conv1: Conv2d
model.sana_vae.encoder.down_blocks.1.0.conv2: Conv2d
model.sana_vae.encoder.down_blocks.1.0.norm: RMSNorm
model.sana_vae.encoder.down_blocks.1.1: ResBlock
model.sana_vae.encoder.down_blocks.1.1.nonlinearity: SiLU
model.sana_vae.encoder.down_blocks.1.1.conv1: Conv2d
model.sana_vae.encoder.down_blocks.1.1.conv2: Conv2d
model.sana_vae.encoder.down_blocks.1.1.norm: RMSNorm
model.sana_vae.encoder.down_blocks.1.2: DCDownBlock2d
model.sana_vae.encoder.down_blocks.1.2.conv: Conv2d
model.sana_vae.encoder.down_blocks.2: Sequential
model.sana_vae.encoder.down_blocks.2.0: ResBlock
model.sana_vae.encoder.down_blocks.2.0.nonlinearity: SiLU
model.sana_vae.encoder.down_blocks.2.0.conv1: Conv2d
model.sana_vae.encoder.down_blocks.2.0.conv2: Conv2d
model.sana_vae.encoder.down_blocks.2.0.norm: RMSNorm
model.sana_vae.encoder.down_blocks.2.1: ResBlock
model.sana_vae.encoder.down_blocks.2.1.nonlinearity: SiLU
model.sana_vae.encoder.down_blocks.2.1.conv1: Conv2d
model.sana_vae.encoder.down_blocks.2.1.conv2: Conv2d
model.sana_vae.encoder.down_blocks.2.1.norm: RMSNorm
model.sana_vae.encoder.down_blocks.2.2: DCDownBlock2d
model.sana_vae.encoder.down_blocks.2.2.conv: Conv2d
model.sana_vae.encoder.down_blocks.3: Sequential
model.sana_vae.encoder.down_blocks.3.0: EfficientViTBlock
model.sana_vae.encoder.down_blocks.3.0.attn: SanaMultiscaleLinearAttention
model.sana_vae.encoder.down_blocks.3.0.attn.to_q: Linear
model.sana_vae.encoder.down_blocks.3.0.attn.to_k: Linear
model.sana_vae.encoder.down_blocks.3.0.attn.to_v: Linear
model.sana_vae.encoder.down_blocks.3.0.attn.to_qkv_multiscale: ModuleList
model.sana_vae.encoder.down_blocks.3.0.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.encoder.down_blocks.3.0.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.encoder.down_blocks.3.0.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.encoder.down_blocks.3.0.attn.nonlinearity: ReLU
model.sana_vae.encoder.down_blocks.3.0.attn.to_out: Linear
model.sana_vae.encoder.down_blocks.3.0.attn.norm_out: RMSNorm
model.sana_vae.encoder.down_blocks.3.0.conv_out: GLUMBConv
model.sana_vae.encoder.down_blocks.3.0.conv_out.nonlinearity: SiLU
model.sana_vae.encoder.down_blocks.3.0.conv_out.conv_inverted: Conv2d
model.sana_vae.encoder.down_blocks.3.0.conv_out.conv_depth: Conv2d
model.sana_vae.encoder.down_blocks.3.0.conv_out.conv_point: Conv2d
model.sana_vae.encoder.down_blocks.3.0.conv_out.norm: RMSNorm
model.sana_vae.encoder.down_blocks.3.1: EfficientViTBlock
model.sana_vae.encoder.down_blocks.3.1.attn: SanaMultiscaleLinearAttention
model.sana_vae.encoder.down_blocks.3.1.attn.to_q: Linear
model.sana_vae.encoder.down_blocks.3.1.attn.to_k: Linear
model.sana_vae.encoder.down_blocks.3.1.attn.to_v: Linear
model.sana_vae.encoder.down_blocks.3.1.attn.to_qkv_multiscale: ModuleList
model.sana_vae.encoder.down_blocks.3.1.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.encoder.down_blocks.3.1.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.encoder.down_blocks.3.1.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.encoder.down_blocks.3.1.attn.nonlinearity: ReLU
model.sana_vae.encoder.down_blocks.3.1.attn.to_out: Linear
model.sana_vae.encoder.down_blocks.3.1.attn.norm_out: RMSNorm
model.sana_vae.encoder.down_blocks.3.1.conv_out: GLUMBConv
model.sana_vae.encoder.down_blocks.3.1.conv_out.nonlinearity: SiLU
model.sana_vae.encoder.down_blocks.3.1.conv_out.conv_inverted: Conv2d
model.sana_vae.encoder.down_blocks.3.1.conv_out.conv_depth: Conv2d
model.sana_vae.encoder.down_blocks.3.1.conv_out.conv_point: Conv2d
model.sana_vae.encoder.down_blocks.3.1.conv_out.norm: RMSNorm
model.sana_vae.encoder.down_blocks.3.2: EfficientViTBlock
model.sana_vae.encoder.down_blocks.3.2.attn: SanaMultiscaleLinearAttention
model.sana_vae.encoder.down_blocks.3.2.attn.to_q: Linear
model.sana_vae.encoder.down_blocks.3.2.attn.to_k: Linear
model.sana_vae.encoder.down_blocks.3.2.attn.to_v: Linear
model.sana_vae.encoder.down_blocks.3.2.attn.to_qkv_multiscale: ModuleList
model.sana_vae.encoder.down_blocks.3.2.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.encoder.down_blocks.3.2.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.encoder.down_blocks.3.2.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.encoder.down_blocks.3.2.attn.nonlinearity: ReLU
model.sana_vae.encoder.down_blocks.3.2.attn.to_out: Linear
model.sana_vae.encoder.down_blocks.3.2.attn.norm_out: RMSNorm
model.sana_vae.encoder.down_blocks.3.2.conv_out: GLUMBConv
model.sana_vae.encoder.down_blocks.3.2.conv_out.nonlinearity: SiLU
model.sana_vae.encoder.down_blocks.3.2.conv_out.conv_inverted: Conv2d
model.sana_vae.encoder.down_blocks.3.2.conv_out.conv_depth: Conv2d
model.sana_vae.encoder.down_blocks.3.2.conv_out.conv_point: Conv2d
model.sana_vae.encoder.down_blocks.3.2.conv_out.norm: RMSNorm
model.sana_vae.encoder.down_blocks.3.3: DCDownBlock2d
model.sana_vae.encoder.down_blocks.3.3.conv: Conv2d
model.sana_vae.encoder.down_blocks.4: Sequential
model.sana_vae.encoder.down_blocks.4.0: EfficientViTBlock
model.sana_vae.encoder.down_blocks.4.0.attn: SanaMultiscaleLinearAttention
model.sana_vae.encoder.down_blocks.4.0.attn.to_q: Linear
model.sana_vae.encoder.down_blocks.4.0.attn.to_k: Linear
model.sana_vae.encoder.down_blocks.4.0.attn.to_v: Linear
model.sana_vae.encoder.down_blocks.4.0.attn.to_qkv_multiscale: ModuleList
model.sana_vae.encoder.down_blocks.4.0.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.encoder.down_blocks.4.0.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.encoder.down_blocks.4.0.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.encoder.down_blocks.4.0.attn.nonlinearity: ReLU
model.sana_vae.encoder.down_blocks.4.0.attn.to_out: Linear
model.sana_vae.encoder.down_blocks.4.0.attn.norm_out: RMSNorm
model.sana_vae.encoder.down_blocks.4.0.conv_out: GLUMBConv
model.sana_vae.encoder.down_blocks.4.0.conv_out.nonlinearity: SiLU
model.sana_vae.encoder.down_blocks.4.0.conv_out.conv_inverted: Conv2d
model.sana_vae.encoder.down_blocks.4.0.conv_out.conv_depth: Conv2d
model.sana_vae.encoder.down_blocks.4.0.conv_out.conv_point: Conv2d
model.sana_vae.encoder.down_blocks.4.0.conv_out.norm: RMSNorm
model.sana_vae.encoder.down_blocks.4.1: EfficientViTBlock
model.sana_vae.encoder.down_blocks.4.1.attn: SanaMultiscaleLinearAttention
model.sana_vae.encoder.down_blocks.4.1.attn.to_q: Linear
model.sana_vae.encoder.down_blocks.4.1.attn.to_k: Linear
model.sana_vae.encoder.down_blocks.4.1.attn.to_v: Linear
model.sana_vae.encoder.down_blocks.4.1.attn.to_qkv_multiscale: ModuleList
model.sana_vae.encoder.down_blocks.4.1.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.encoder.down_blocks.4.1.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.encoder.down_blocks.4.1.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.encoder.down_blocks.4.1.attn.nonlinearity: ReLU
model.sana_vae.encoder.down_blocks.4.1.attn.to_out: Linear
model.sana_vae.encoder.down_blocks.4.1.attn.norm_out: RMSNorm
model.sana_vae.encoder.down_blocks.4.1.conv_out: GLUMBConv
model.sana_vae.encoder.down_blocks.4.1.conv_out.nonlinearity: SiLU
model.sana_vae.encoder.down_blocks.4.1.conv_out.conv_inverted: Conv2d
model.sana_vae.encoder.down_blocks.4.1.conv_out.conv_depth: Conv2d
model.sana_vae.encoder.down_blocks.4.1.conv_out.conv_point: Conv2d
model.sana_vae.encoder.down_blocks.4.1.conv_out.norm: RMSNorm
model.sana_vae.encoder.down_blocks.4.2: EfficientViTBlock
model.sana_vae.encoder.down_blocks.4.2.attn: SanaMultiscaleLinearAttention
model.sana_vae.encoder.down_blocks.4.2.attn.to_q: Linear
model.sana_vae.encoder.down_blocks.4.2.attn.to_k: Linear
model.sana_vae.encoder.down_blocks.4.2.attn.to_v: Linear
model.sana_vae.encoder.down_blocks.4.2.attn.to_qkv_multiscale: ModuleList
model.sana_vae.encoder.down_blocks.4.2.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.encoder.down_blocks.4.2.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.encoder.down_blocks.4.2.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.encoder.down_blocks.4.2.attn.nonlinearity: ReLU
model.sana_vae.encoder.down_blocks.4.2.attn.to_out: Linear
model.sana_vae.encoder.down_blocks.4.2.attn.norm_out: RMSNorm
model.sana_vae.encoder.down_blocks.4.2.conv_out: GLUMBConv
model.sana_vae.encoder.down_blocks.4.2.conv_out.nonlinearity: SiLU
model.sana_vae.encoder.down_blocks.4.2.conv_out.conv_inverted: Conv2d
model.sana_vae.encoder.down_blocks.4.2.conv_out.conv_depth: Conv2d
model.sana_vae.encoder.down_blocks.4.2.conv_out.conv_point: Conv2d
model.sana_vae.encoder.down_blocks.4.2.conv_out.norm: RMSNorm
model.sana_vae.encoder.down_blocks.4.3: DCDownBlock2d
model.sana_vae.encoder.down_blocks.4.3.conv: Conv2d
model.sana_vae.encoder.down_blocks.5: Sequential
model.sana_vae.encoder.down_blocks.5.0: EfficientViTBlock
model.sana_vae.encoder.down_blocks.5.0.attn: SanaMultiscaleLinearAttention
model.sana_vae.encoder.down_blocks.5.0.attn.to_q: Linear
model.sana_vae.encoder.down_blocks.5.0.attn.to_k: Linear
model.sana_vae.encoder.down_blocks.5.0.attn.to_v: Linear
model.sana_vae.encoder.down_blocks.5.0.attn.to_qkv_multiscale: ModuleList
model.sana_vae.encoder.down_blocks.5.0.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.encoder.down_blocks.5.0.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.encoder.down_blocks.5.0.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.encoder.down_blocks.5.0.attn.nonlinearity: ReLU
model.sana_vae.encoder.down_blocks.5.0.attn.to_out: Linear
model.sana_vae.encoder.down_blocks.5.0.attn.norm_out: RMSNorm
model.sana_vae.encoder.down_blocks.5.0.conv_out: GLUMBConv
model.sana_vae.encoder.down_blocks.5.0.conv_out.nonlinearity: SiLU
model.sana_vae.encoder.down_blocks.5.0.conv_out.conv_inverted: Conv2d
model.sana_vae.encoder.down_blocks.5.0.conv_out.conv_depth: Conv2d
model.sana_vae.encoder.down_blocks.5.0.conv_out.conv_point: Conv2d
model.sana_vae.encoder.down_blocks.5.0.conv_out.norm: RMSNorm
model.sana_vae.encoder.down_blocks.5.1: EfficientViTBlock
model.sana_vae.encoder.down_blocks.5.1.attn: SanaMultiscaleLinearAttention
model.sana_vae.encoder.down_blocks.5.1.attn.to_q: Linear
model.sana_vae.encoder.down_blocks.5.1.attn.to_k: Linear
model.sana_vae.encoder.down_blocks.5.1.attn.to_v: Linear
model.sana_vae.encoder.down_blocks.5.1.attn.to_qkv_multiscale: ModuleList
model.sana_vae.encoder.down_blocks.5.1.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.encoder.down_blocks.5.1.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.encoder.down_blocks.5.1.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.encoder.down_blocks.5.1.attn.nonlinearity: ReLU
model.sana_vae.encoder.down_blocks.5.1.attn.to_out: Linear
model.sana_vae.encoder.down_blocks.5.1.attn.norm_out: RMSNorm
model.sana_vae.encoder.down_blocks.5.1.conv_out: GLUMBConv
model.sana_vae.encoder.down_blocks.5.1.conv_out.nonlinearity: SiLU
model.sana_vae.encoder.down_blocks.5.1.conv_out.conv_inverted: Conv2d
model.sana_vae.encoder.down_blocks.5.1.conv_out.conv_depth: Conv2d
model.sana_vae.encoder.down_blocks.5.1.conv_out.conv_point: Conv2d
model.sana_vae.encoder.down_blocks.5.1.conv_out.norm: RMSNorm
model.sana_vae.encoder.down_blocks.5.2: EfficientViTBlock
model.sana_vae.encoder.down_blocks.5.2.attn: SanaMultiscaleLinearAttention
model.sana_vae.encoder.down_blocks.5.2.attn.to_q: Linear
model.sana_vae.encoder.down_blocks.5.2.attn.to_k: Linear
model.sana_vae.encoder.down_blocks.5.2.attn.to_v: Linear
model.sana_vae.encoder.down_blocks.5.2.attn.to_qkv_multiscale: ModuleList
model.sana_vae.encoder.down_blocks.5.2.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.encoder.down_blocks.5.2.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.encoder.down_blocks.5.2.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.encoder.down_blocks.5.2.attn.nonlinearity: ReLU
model.sana_vae.encoder.down_blocks.5.2.attn.to_out: Linear
model.sana_vae.encoder.down_blocks.5.2.attn.norm_out: RMSNorm
model.sana_vae.encoder.down_blocks.5.2.conv_out: GLUMBConv
model.sana_vae.encoder.down_blocks.5.2.conv_out.nonlinearity: SiLU
model.sana_vae.encoder.down_blocks.5.2.conv_out.conv_inverted: Conv2d
model.sana_vae.encoder.down_blocks.5.2.conv_out.conv_depth: Conv2d
model.sana_vae.encoder.down_blocks.5.2.conv_out.conv_point: Conv2d
model.sana_vae.encoder.down_blocks.5.2.conv_out.norm: RMSNorm
model.sana_vae.encoder.conv_out: Conv2d
model.sana_vae.decoder: Decoder
model.sana_vae.decoder.conv_in: Conv2d
model.sana_vae.decoder.up_blocks: ModuleList
model.sana_vae.decoder.up_blocks.0: Sequential
model.sana_vae.decoder.up_blocks.0.0: DCUpBlock2d
model.sana_vae.decoder.up_blocks.0.0.conv: Conv2d
model.sana_vae.decoder.up_blocks.0.1: ResBlock
model.sana_vae.decoder.up_blocks.0.1.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.0.1.conv1: Conv2d
model.sana_vae.decoder.up_blocks.0.1.conv2: Conv2d
model.sana_vae.decoder.up_blocks.0.1.norm: RMSNorm
model.sana_vae.decoder.up_blocks.0.2: ResBlock
model.sana_vae.decoder.up_blocks.0.2.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.0.2.conv1: Conv2d
model.sana_vae.decoder.up_blocks.0.2.conv2: Conv2d
model.sana_vae.decoder.up_blocks.0.2.norm: RMSNorm
model.sana_vae.decoder.up_blocks.0.3: ResBlock
model.sana_vae.decoder.up_blocks.0.3.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.0.3.conv1: Conv2d
model.sana_vae.decoder.up_blocks.0.3.conv2: Conv2d
model.sana_vae.decoder.up_blocks.0.3.norm: RMSNorm
model.sana_vae.decoder.up_blocks.1: Sequential
model.sana_vae.decoder.up_blocks.1.0: DCUpBlock2d
model.sana_vae.decoder.up_blocks.1.0.conv: Conv2d
model.sana_vae.decoder.up_blocks.1.1: ResBlock
model.sana_vae.decoder.up_blocks.1.1.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.1.1.conv1: Conv2d
model.sana_vae.decoder.up_blocks.1.1.conv2: Conv2d
model.sana_vae.decoder.up_blocks.1.1.norm: RMSNorm
model.sana_vae.decoder.up_blocks.1.2: ResBlock
model.sana_vae.decoder.up_blocks.1.2.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.1.2.conv1: Conv2d
model.sana_vae.decoder.up_blocks.1.2.conv2: Conv2d
model.sana_vae.decoder.up_blocks.1.2.norm: RMSNorm
model.sana_vae.decoder.up_blocks.1.3: ResBlock
model.sana_vae.decoder.up_blocks.1.3.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.1.3.conv1: Conv2d
model.sana_vae.decoder.up_blocks.1.3.conv2: Conv2d
model.sana_vae.decoder.up_blocks.1.3.norm: RMSNorm
model.sana_vae.decoder.up_blocks.2: Sequential
model.sana_vae.decoder.up_blocks.2.0: DCUpBlock2d
model.sana_vae.decoder.up_blocks.2.0.conv: Conv2d
model.sana_vae.decoder.up_blocks.2.1: ResBlock
model.sana_vae.decoder.up_blocks.2.1.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.2.1.conv1: Conv2d
model.sana_vae.decoder.up_blocks.2.1.conv2: Conv2d
model.sana_vae.decoder.up_blocks.2.1.norm: RMSNorm
model.sana_vae.decoder.up_blocks.2.2: ResBlock
model.sana_vae.decoder.up_blocks.2.2.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.2.2.conv1: Conv2d
model.sana_vae.decoder.up_blocks.2.2.conv2: Conv2d
model.sana_vae.decoder.up_blocks.2.2.norm: RMSNorm
model.sana_vae.decoder.up_blocks.2.3: ResBlock
model.sana_vae.decoder.up_blocks.2.3.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.2.3.conv1: Conv2d
model.sana_vae.decoder.up_blocks.2.3.conv2: Conv2d
model.sana_vae.decoder.up_blocks.2.3.norm: RMSNorm
model.sana_vae.decoder.up_blocks.3: Sequential
model.sana_vae.decoder.up_blocks.3.0: DCUpBlock2d
model.sana_vae.decoder.up_blocks.3.0.conv: Conv2d
model.sana_vae.decoder.up_blocks.3.1: EfficientViTBlock
model.sana_vae.decoder.up_blocks.3.1.attn: SanaMultiscaleLinearAttention
model.sana_vae.decoder.up_blocks.3.1.attn.to_q: Linear
model.sana_vae.decoder.up_blocks.3.1.attn.to_k: Linear
model.sana_vae.decoder.up_blocks.3.1.attn.to_v: Linear
model.sana_vae.decoder.up_blocks.3.1.attn.to_qkv_multiscale: ModuleList
model.sana_vae.decoder.up_blocks.3.1.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.decoder.up_blocks.3.1.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.decoder.up_blocks.3.1.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.decoder.up_blocks.3.1.attn.nonlinearity: ReLU
model.sana_vae.decoder.up_blocks.3.1.attn.to_out: Linear
model.sana_vae.decoder.up_blocks.3.1.attn.norm_out: RMSNorm
model.sana_vae.decoder.up_blocks.3.1.conv_out: GLUMBConv
model.sana_vae.decoder.up_blocks.3.1.conv_out.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.3.1.conv_out.conv_inverted: Conv2d
model.sana_vae.decoder.up_blocks.3.1.conv_out.conv_depth: Conv2d
model.sana_vae.decoder.up_blocks.3.1.conv_out.conv_point: Conv2d
model.sana_vae.decoder.up_blocks.3.1.conv_out.norm: RMSNorm
model.sana_vae.decoder.up_blocks.3.2: EfficientViTBlock
model.sana_vae.decoder.up_blocks.3.2.attn: SanaMultiscaleLinearAttention
model.sana_vae.decoder.up_blocks.3.2.attn.to_q: Linear
model.sana_vae.decoder.up_blocks.3.2.attn.to_k: Linear
model.sana_vae.decoder.up_blocks.3.2.attn.to_v: Linear
model.sana_vae.decoder.up_blocks.3.2.attn.to_qkv_multiscale: ModuleList
model.sana_vae.decoder.up_blocks.3.2.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.decoder.up_blocks.3.2.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.decoder.up_blocks.3.2.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.decoder.up_blocks.3.2.attn.nonlinearity: ReLU
model.sana_vae.decoder.up_blocks.3.2.attn.to_out: Linear
model.sana_vae.decoder.up_blocks.3.2.attn.norm_out: RMSNorm
model.sana_vae.decoder.up_blocks.3.2.conv_out: GLUMBConv
model.sana_vae.decoder.up_blocks.3.2.conv_out.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.3.2.conv_out.conv_inverted: Conv2d
model.sana_vae.decoder.up_blocks.3.2.conv_out.conv_depth: Conv2d
model.sana_vae.decoder.up_blocks.3.2.conv_out.conv_point: Conv2d
model.sana_vae.decoder.up_blocks.3.2.conv_out.norm: RMSNorm
model.sana_vae.decoder.up_blocks.3.3: EfficientViTBlock
model.sana_vae.decoder.up_blocks.3.3.attn: SanaMultiscaleLinearAttention
model.sana_vae.decoder.up_blocks.3.3.attn.to_q: Linear
model.sana_vae.decoder.up_blocks.3.3.attn.to_k: Linear
model.sana_vae.decoder.up_blocks.3.3.attn.to_v: Linear
model.sana_vae.decoder.up_blocks.3.3.attn.to_qkv_multiscale: ModuleList
model.sana_vae.decoder.up_blocks.3.3.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.decoder.up_blocks.3.3.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.decoder.up_blocks.3.3.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.decoder.up_blocks.3.3.attn.nonlinearity: ReLU
model.sana_vae.decoder.up_blocks.3.3.attn.to_out: Linear
model.sana_vae.decoder.up_blocks.3.3.attn.norm_out: RMSNorm
model.sana_vae.decoder.up_blocks.3.3.conv_out: GLUMBConv
model.sana_vae.decoder.up_blocks.3.3.conv_out.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.3.3.conv_out.conv_inverted: Conv2d
model.sana_vae.decoder.up_blocks.3.3.conv_out.conv_depth: Conv2d
model.sana_vae.decoder.up_blocks.3.3.conv_out.conv_point: Conv2d
model.sana_vae.decoder.up_blocks.3.3.conv_out.norm: RMSNorm
model.sana_vae.decoder.up_blocks.4: Sequential
model.sana_vae.decoder.up_blocks.4.0: DCUpBlock2d
model.sana_vae.decoder.up_blocks.4.0.conv: Conv2d
model.sana_vae.decoder.up_blocks.4.1: EfficientViTBlock
model.sana_vae.decoder.up_blocks.4.1.attn: SanaMultiscaleLinearAttention
model.sana_vae.decoder.up_blocks.4.1.attn.to_q: Linear
model.sana_vae.decoder.up_blocks.4.1.attn.to_k: Linear
model.sana_vae.decoder.up_blocks.4.1.attn.to_v: Linear
model.sana_vae.decoder.up_blocks.4.1.attn.to_qkv_multiscale: ModuleList
model.sana_vae.decoder.up_blocks.4.1.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.decoder.up_blocks.4.1.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.decoder.up_blocks.4.1.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.decoder.up_blocks.4.1.attn.nonlinearity: ReLU
model.sana_vae.decoder.up_blocks.4.1.attn.to_out: Linear
model.sana_vae.decoder.up_blocks.4.1.attn.norm_out: RMSNorm
model.sana_vae.decoder.up_blocks.4.1.conv_out: GLUMBConv
model.sana_vae.decoder.up_blocks.4.1.conv_out.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.4.1.conv_out.conv_inverted: Conv2d
model.sana_vae.decoder.up_blocks.4.1.conv_out.conv_depth: Conv2d
model.sana_vae.decoder.up_blocks.4.1.conv_out.conv_point: Conv2d
model.sana_vae.decoder.up_blocks.4.1.conv_out.norm: RMSNorm
model.sana_vae.decoder.up_blocks.4.2: EfficientViTBlock
model.sana_vae.decoder.up_blocks.4.2.attn: SanaMultiscaleLinearAttention
model.sana_vae.decoder.up_blocks.4.2.attn.to_q: Linear
model.sana_vae.decoder.up_blocks.4.2.attn.to_k: Linear
model.sana_vae.decoder.up_blocks.4.2.attn.to_v: Linear
model.sana_vae.decoder.up_blocks.4.2.attn.to_qkv_multiscale: ModuleList
model.sana_vae.decoder.up_blocks.4.2.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.decoder.up_blocks.4.2.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.decoder.up_blocks.4.2.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.decoder.up_blocks.4.2.attn.nonlinearity: ReLU
model.sana_vae.decoder.up_blocks.4.2.attn.to_out: Linear
model.sana_vae.decoder.up_blocks.4.2.attn.norm_out: RMSNorm
model.sana_vae.decoder.up_blocks.4.2.conv_out: GLUMBConv
model.sana_vae.decoder.up_blocks.4.2.conv_out.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.4.2.conv_out.conv_inverted: Conv2d
model.sana_vae.decoder.up_blocks.4.2.conv_out.conv_depth: Conv2d
model.sana_vae.decoder.up_blocks.4.2.conv_out.conv_point: Conv2d
model.sana_vae.decoder.up_blocks.4.2.conv_out.norm: RMSNorm
model.sana_vae.decoder.up_blocks.4.3: EfficientViTBlock
model.sana_vae.decoder.up_blocks.4.3.attn: SanaMultiscaleLinearAttention
model.sana_vae.decoder.up_blocks.4.3.attn.to_q: Linear
model.sana_vae.decoder.up_blocks.4.3.attn.to_k: Linear
model.sana_vae.decoder.up_blocks.4.3.attn.to_v: Linear
model.sana_vae.decoder.up_blocks.4.3.attn.to_qkv_multiscale: ModuleList
model.sana_vae.decoder.up_blocks.4.3.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.decoder.up_blocks.4.3.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.decoder.up_blocks.4.3.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.decoder.up_blocks.4.3.attn.nonlinearity: ReLU
model.sana_vae.decoder.up_blocks.4.3.attn.to_out: Linear
model.sana_vae.decoder.up_blocks.4.3.attn.norm_out: RMSNorm
model.sana_vae.decoder.up_blocks.4.3.conv_out: GLUMBConv
model.sana_vae.decoder.up_blocks.4.3.conv_out.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.4.3.conv_out.conv_inverted: Conv2d
model.sana_vae.decoder.up_blocks.4.3.conv_out.conv_depth: Conv2d
model.sana_vae.decoder.up_blocks.4.3.conv_out.conv_point: Conv2d
model.sana_vae.decoder.up_blocks.4.3.conv_out.norm: RMSNorm
model.sana_vae.decoder.up_blocks.5: Sequential
model.sana_vae.decoder.up_blocks.5.0: EfficientViTBlock
model.sana_vae.decoder.up_blocks.5.0.attn: SanaMultiscaleLinearAttention
model.sana_vae.decoder.up_blocks.5.0.attn.to_q: Linear
model.sana_vae.decoder.up_blocks.5.0.attn.to_k: Linear
model.sana_vae.decoder.up_blocks.5.0.attn.to_v: Linear
model.sana_vae.decoder.up_blocks.5.0.attn.to_qkv_multiscale: ModuleList
model.sana_vae.decoder.up_blocks.5.0.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.decoder.up_blocks.5.0.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.decoder.up_blocks.5.0.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.decoder.up_blocks.5.0.attn.nonlinearity: ReLU
model.sana_vae.decoder.up_blocks.5.0.attn.to_out: Linear
model.sana_vae.decoder.up_blocks.5.0.attn.norm_out: RMSNorm
model.sana_vae.decoder.up_blocks.5.0.conv_out: GLUMBConv
model.sana_vae.decoder.up_blocks.5.0.conv_out.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.5.0.conv_out.conv_inverted: Conv2d
model.sana_vae.decoder.up_blocks.5.0.conv_out.conv_depth: Conv2d
model.sana_vae.decoder.up_blocks.5.0.conv_out.conv_point: Conv2d
model.sana_vae.decoder.up_blocks.5.0.conv_out.norm: RMSNorm
model.sana_vae.decoder.up_blocks.5.1: EfficientViTBlock
model.sana_vae.decoder.up_blocks.5.1.attn: SanaMultiscaleLinearAttention
model.sana_vae.decoder.up_blocks.5.1.attn.to_q: Linear
model.sana_vae.decoder.up_blocks.5.1.attn.to_k: Linear
model.sana_vae.decoder.up_blocks.5.1.attn.to_v: Linear
model.sana_vae.decoder.up_blocks.5.1.attn.to_qkv_multiscale: ModuleList
model.sana_vae.decoder.up_blocks.5.1.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.decoder.up_blocks.5.1.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.decoder.up_blocks.5.1.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.decoder.up_blocks.5.1.attn.nonlinearity: ReLU
model.sana_vae.decoder.up_blocks.5.1.attn.to_out: Linear
model.sana_vae.decoder.up_blocks.5.1.attn.norm_out: RMSNorm
model.sana_vae.decoder.up_blocks.5.1.conv_out: GLUMBConv
model.sana_vae.decoder.up_blocks.5.1.conv_out.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.5.1.conv_out.conv_inverted: Conv2d
model.sana_vae.decoder.up_blocks.5.1.conv_out.conv_depth: Conv2d
model.sana_vae.decoder.up_blocks.5.1.conv_out.conv_point: Conv2d
model.sana_vae.decoder.up_blocks.5.1.conv_out.norm: RMSNorm
model.sana_vae.decoder.up_blocks.5.2: EfficientViTBlock
model.sana_vae.decoder.up_blocks.5.2.attn: SanaMultiscaleLinearAttention
model.sana_vae.decoder.up_blocks.5.2.attn.to_q: Linear
model.sana_vae.decoder.up_blocks.5.2.attn.to_k: Linear
model.sana_vae.decoder.up_blocks.5.2.attn.to_v: Linear
model.sana_vae.decoder.up_blocks.5.2.attn.to_qkv_multiscale: ModuleList
model.sana_vae.decoder.up_blocks.5.2.attn.to_qkv_multiscale.0: SanaMultiscaleAttentionProjection
model.sana_vae.decoder.up_blocks.5.2.attn.to_qkv_multiscale.0.proj_in: Conv2d
model.sana_vae.decoder.up_blocks.5.2.attn.to_qkv_multiscale.0.proj_out: Conv2d
model.sana_vae.decoder.up_blocks.5.2.attn.nonlinearity: ReLU
model.sana_vae.decoder.up_blocks.5.2.attn.to_out: Linear
model.sana_vae.decoder.up_blocks.5.2.attn.norm_out: RMSNorm
model.sana_vae.decoder.up_blocks.5.2.conv_out: GLUMBConv
model.sana_vae.decoder.up_blocks.5.2.conv_out.nonlinearity: SiLU
model.sana_vae.decoder.up_blocks.5.2.conv_out.conv_inverted: Conv2d
model.sana_vae.decoder.up_blocks.5.2.conv_out.conv_depth: Conv2d
model.sana_vae.decoder.up_blocks.5.2.conv_out.conv_point: Conv2d
model.sana_vae.decoder.up_blocks.5.2.conv_out.norm: RMSNorm
model.sana_vae.decoder.norm_out: RMSNorm
model.sana_vae.decoder.conv_act: ReLU
model.sana_vae.decoder.conv_out: Conv2d
model.diffusion_connector: Sequential
model.diffusion_connector.0: Linear
model.diffusion_connector.1: GELU
model.diffusion_connector.2: Linear
model.diffusion_connector.3: RMSNorm
model.vae_connector: Sequential
model.vae_connector.0: Linear
model.vae_connector.1: GELU
model.vae_connector.2: Linear
lm_head: Linear

3. Parameter Statistics
--------------------------------------------------
Total Parameters: 4,382,690,307
Trainable Parameters: 3,791,658,307
Frozen Parameters: 591,032,000
Trainable Parameter Ratio: 86.51%

4. Module Parameter Details
--------------------------------------------------

model.embed_tokens:
  Total Parameters: 444,848,128
  Trainable: 444,848,128
  Frozen: 0
  Trainable ratio: 100.00%

lm_head:
  Total Parameters: 444,848,128
  Trainable: 444,848,128
  Frozen: 0
  Trainable ratio: 100.00%

model.vision_tower.vision_tower.bottleneck.regularizer.embedding:
  Total Parameters: 100,663,296
  Trainable: 0
  Frozen: 100,663,296
  Trainable ratio: 0.00%

model.sana.time_embed.linear:
  Total Parameters: 30,119,040
  Trainable: 30,119,040
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.0.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.1.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.2.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.3.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.4.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.5.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.6.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.7.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.8.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.9.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.10.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.11.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.12.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.13.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.14.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.15.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.16.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.17.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.18.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.19.ff.conv_inverted:
  Total Parameters: 25,099,200
  Trainable: 25,099,200
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.0.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.0.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.0.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.1.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.1.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.1.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.2.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.2.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.2.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.3.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.3.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.3.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.4.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.4.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.4.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.5.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.5.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.5.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.6.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.6.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.6.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.7.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.7.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.7.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.8.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.8.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.8.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.9.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.9.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.9.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.10.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.10.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.10.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.11.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.11.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.11.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.12.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.12.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.12.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.13.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.13.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.13.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.14.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.14.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.14.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.15.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.15.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.15.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.16.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.16.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.16.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.17.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.17.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.17.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.18.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.18.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.18.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.19.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.19.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.19.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.20.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.20.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.20.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.21.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.21.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.21.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.22.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.22.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.22.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.23.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.23.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.23.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.24.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.24.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.24.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.25.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.25.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.25.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.26.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.26.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.26.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.27.mlp.gate_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.27.mlp.up_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.27.mlp.down_proj:
  Total Parameters: 12,582,912
  Trainable: 12,582,912
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.0.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.1.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.2.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.3.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.4.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.5.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.6.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.7.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.8.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.9.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.10.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.11.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.12.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.13.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.14.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.15.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.16.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.17.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.18.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.19.ff.conv_point:
  Total Parameters: 12,544,000
  Trainable: 12,544,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.3.conv:
  Total Parameters: 9,438,208
  Trainable: 9,438,208
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.0.conv:
  Total Parameters: 9,438,208
  Trainable: 9,438,208
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.0.conv_out.conv_inverted:
  Total Parameters: 8,396,800
  Trainable: 8,396,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.1.conv_out.conv_inverted:
  Total Parameters: 8,396,800
  Trainable: 8,396,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.2.conv_out.conv_inverted:
  Total Parameters: 8,396,800
  Trainable: 8,396,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.0.conv_out.conv_inverted:
  Total Parameters: 8,396,800
  Trainable: 8,396,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.1.conv_out.conv_inverted:
  Total Parameters: 8,396,800
  Trainable: 8,396,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.2.conv_out.conv_inverted:
  Total Parameters: 8,396,800
  Trainable: 8,396,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.1.conv_out.conv_inverted:
  Total Parameters: 8,396,800
  Trainable: 8,396,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.2.conv_out.conv_inverted:
  Total Parameters: 8,396,800
  Trainable: 8,396,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.3.conv_out.conv_inverted:
  Total Parameters: 8,396,800
  Trainable: 8,396,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.0.conv_out.conv_inverted:
  Total Parameters: 8,396,800
  Trainable: 8,396,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.1.conv_out.conv_inverted:
  Total Parameters: 8,396,800
  Trainable: 8,396,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.2.conv_out.conv_inverted:
  Total Parameters: 8,396,800
  Trainable: 8,396,800
  Frozen: 0
  Trainable ratio: 100.00%

model.diffusion_connector.2:
  Total Parameters: 5,310,720
  Trainable: 5,310,720
  Frozen: 0
  Trainable ratio: 100.00%

model.vae_connector.2:
  Total Parameters: 5,310,720
  Trainable: 5,310,720
  Frozen: 0
  Trainable ratio: 100.00%

model.vae_connector.0:
  Total Parameters: 5,163,264
  Trainable: 5,163,264
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.caption_projection.linear_1:
  Total Parameters: 5,163,200
  Trainable: 5,163,200
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.time_embed.emb.timestep_embedder.linear_2:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.caption_projection.linear_2:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.0.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.0.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.0.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.0.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.0.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.1.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.1.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.1.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.1.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.1.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.2.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.2.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.2.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.2.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.2.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.3.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.3.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.3.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.3.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.3.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.4.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.4.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.4.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.4.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.4.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.5.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.5.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.5.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.5.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.5.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.6.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.6.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.6.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.6.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.6.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.7.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.7.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.7.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.7.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.7.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.8.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.8.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.8.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.8.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.8.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.9.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.9.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.9.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.9.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.9.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.10.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.10.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.10.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.10.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.10.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.11.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.11.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.11.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.11.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.11.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.12.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.12.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.12.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.12.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.12.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.13.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.13.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.13.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.13.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.13.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.14.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.14.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.14.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.14.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.14.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.15.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.15.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.15.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.15.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.15.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.16.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.16.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.16.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.16.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.16.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.17.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.17.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.17.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.17.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.17.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.18.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.18.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.18.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.18.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.18.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.19.attn1.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.19.attn2.to_q:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.19.attn2.to_k:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.19.attn2.to_v:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.19.attn2.to_out.0:
  Total Parameters: 5,019,840
  Trainable: 5,019,840
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.0.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.0.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.0.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.1.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.1.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.1.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.2.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.2.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.2.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.3.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.3.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.3.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.4.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.4.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.4.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.5.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.5.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.5.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.6.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.6.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.6.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.7.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.7.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.7.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.8.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.8.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.8.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.9.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.9.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.9.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.10.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.10.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.10.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.11.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.11.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.11.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.12.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.12.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.12.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.13.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.13.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.13.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.14.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.14.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.14.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.15.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.15.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.15.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.16.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.16.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.16.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.17.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.17.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.17.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.18.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.18.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.18.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.19.attn1.to_q:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.19.attn1.to_k:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.19.attn1.to_v:
  Total Parameters: 5,017,600
  Trainable: 5,017,600
  Frozen: 0
  Trainable ratio: 100.00%

model.vision_tower.vision_tower.encoder.encoder.layers.0.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.1.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.2.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.3.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.4.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.5.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.6.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.7.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.8.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.9.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.10.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.11.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.12.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.13.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.14.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.15.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.16.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.17.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.18.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.19.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.20.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.21.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.22.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.23.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.24.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.25.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.26.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.head.mlp.fc1:
  Total Parameters: 4,962,512
  Trainable: 0
  Frozen: 4,962,512
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.0.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.1.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.2.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.3.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.4.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.5.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.6.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.7.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.8.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.9.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.10.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.11.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.12.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.13.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.14.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.15.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.16.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.17.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.18.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.19.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.20.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.21.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.22.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.23.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.24.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.25.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.26.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.head.mlp.fc2:
  Total Parameters: 4,959,360
  Trainable: 0
  Frozen: 4,959,360
  Trainable ratio: 0.00%

model.diffusion_connector.0:
  Total Parameters: 4,720,896
  Trainable: 4,720,896
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.3.conv:
  Total Parameters: 4,719,616
  Trainable: 4,719,616
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.0.conv:
  Total Parameters: 4,719,104
  Trainable: 4,719,104
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.0.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.0.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.1.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.1.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.2.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.2.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.3.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.3.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.4.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.4.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.5.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.5.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.6.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.6.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.7.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.7.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.8.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.8.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.9.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.9.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.10.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.10.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.11.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.11.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.12.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.12.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.13.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.13.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.14.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.14.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.15.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.15.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.16.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.16.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.17.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.17.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.18.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.18.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.19.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.19.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.20.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.20.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.21.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.21.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.22.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.22.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.23.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.23.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.24.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.24.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.25.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.25.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.26.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.26.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.27.self_attn.q_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.27.self_attn.o_proj:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.0.conv_out.conv_point:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.1.conv_out.conv_point:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.2.conv_out.conv_point:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.0.conv_out.conv_point:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.1.conv_out.conv_point:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.2.conv_out.conv_point:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.1.conv_out.conv_point:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.2.conv_out.conv_point:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.3.conv_out.conv_point:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.0.conv_out.conv_point:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.1.conv_out.conv_point:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.2.conv_out.conv_point:
  Total Parameters: 4,194,304
  Trainable: 4,194,304
  Frozen: 0
  Trainable ratio: 100.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.mlp.fc1:
  Total Parameters: 3,542,016
  Trainable: 0
  Frozen: 3,542,016
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.mlp.fc1:
  Total Parameters: 3,542,016
  Trainable: 0
  Frozen: 3,542,016
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.mlp.fc1:
  Total Parameters: 3,542,016
  Trainable: 0
  Frozen: 3,542,016
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.head.mlp.fc1:
  Total Parameters: 3,542,016
  Trainable: 0
  Frozen: 3,542,016
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.mlp.fc2:
  Total Parameters: 3,540,096
  Trainable: 0
  Frozen: 3,540,096
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.mlp.fc2:
  Total Parameters: 3,540,096
  Trainable: 0
  Frozen: 3,540,096
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.mlp.fc2:
  Total Parameters: 3,540,096
  Trainable: 0
  Frozen: 3,540,096
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.head.mlp.fc2:
  Total Parameters: 3,540,096
  Trainable: 0
  Frozen: 3,540,096
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.bottleneck.out_linear:
  Total Parameters: 2,360,832
  Trainable: 0
  Frozen: 2,360,832
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.bottleneck.regularizer.embedding_proj:
  Total Parameters: 2,360,832
  Trainable: 0
  Frozen: 2,360,832
  Trainable ratio: 0.00%

model.sana_vae.encoder.down_blocks.2.0.conv1:
  Total Parameters: 2,359,808
  Trainable: 2,359,808
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.2.1.conv1:
  Total Parameters: 2,359,808
  Trainable: 2,359,808
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.2.2.conv:
  Total Parameters: 2,359,808
  Trainable: 2,359,808
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.2.0.conv:
  Total Parameters: 2,359,808
  Trainable: 2,359,808
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.2.1.conv1:
  Total Parameters: 2,359,808
  Trainable: 2,359,808
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.2.2.conv1:
  Total Parameters: 2,359,808
  Trainable: 2,359,808
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.2.3.conv1:
  Total Parameters: 2,359,808
  Trainable: 2,359,808
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.2.0.conv2:
  Total Parameters: 2,359,296
  Trainable: 2,359,296
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.2.1.conv2:
  Total Parameters: 2,359,296
  Trainable: 2,359,296
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.2.1.conv2:
  Total Parameters: 2,359,296
  Trainable: 2,359,296
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.2.2.conv2:
  Total Parameters: 2,359,296
  Trainable: 2,359,296
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.2.3.conv2:
  Total Parameters: 2,359,296
  Trainable: 2,359,296
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.0.conv_out.conv_inverted:
  Total Parameters: 2,101,248
  Trainable: 2,101,248
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.1.conv_out.conv_inverted:
  Total Parameters: 2,101,248
  Trainable: 2,101,248
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.2.conv_out.conv_inverted:
  Total Parameters: 2,101,248
  Trainable: 2,101,248
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.1.conv_out.conv_inverted:
  Total Parameters: 2,101,248
  Trainable: 2,101,248
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.2.conv_out.conv_inverted:
  Total Parameters: 2,101,248
  Trainable: 2,101,248
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.3.conv_out.conv_inverted:
  Total Parameters: 2,101,248
  Trainable: 2,101,248
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.0.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.0.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.1.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.1.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.2.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.2.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.3.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.3.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.4.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.4.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.5.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.5.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.6.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.6.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.7.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.7.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.8.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.8.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.9.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.9.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.10.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.10.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.11.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.11.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.12.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.12.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.13.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.13.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.14.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.14.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.15.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.15.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.16.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.16.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.17.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.17.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.18.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.18.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.19.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.19.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.20.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.20.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.21.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.21.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.22.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.22.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.23.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.23.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.24.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.24.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.25.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.25.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.26.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.26.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.27.self_attn.k_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.27.self_attn.v_proj:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.0.attn.to_out:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.1.attn.to_out:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.2.attn.to_out:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.0.attn.to_out:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.1.attn.to_out:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.2.attn.to_out:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.1.attn.to_out:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.2.attn.to_out:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.3.attn.to_out:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.0.attn.to_out:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.1.attn.to_out:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.2.attn.to_out:
  Total Parameters: 2,097,152
  Trainable: 2,097,152
  Frozen: 0
  Trainable ratio: 100.00%

model.vision_tower.vision_tower.bottleneck.in_linear:
  Total Parameters: 1,771,008
  Trainable: 0
  Frozen: 1,771,008
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.embeddings.patch_embedding:
  Total Parameters: 1,770,624
  Trainable: 0
  Frozen: 1,770,624
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.0.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.0.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.0.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.0.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.1.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.1.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.1.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.1.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.2.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.2.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.2.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.2.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.3.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.3.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.3.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.3.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.4.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.4.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.4.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.4.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.5.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.5.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.5.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.5.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.6.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.6.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.6.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.6.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.7.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.7.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.7.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.7.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.8.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.8.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.8.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.8.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.9.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.9.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.9.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.9.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.10.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.10.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.10.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.10.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.11.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.11.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.11.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.11.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.12.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.12.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.12.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.12.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.13.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.13.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.13.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.13.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.14.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.14.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.14.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.14.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.15.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.15.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.15.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.15.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.16.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.16.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.16.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.16.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.17.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.17.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.17.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.17.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.18.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.18.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.18.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.18.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.19.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.19.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.19.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.19.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.20.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.20.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.20.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.20.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.21.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.21.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.21.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.21.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.22.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.22.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.22.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.22.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.23.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.23.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.23.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.23.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.24.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.24.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.24.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.24.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.25.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.25.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.25.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.25.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.26.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.26.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.26.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.26.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.head.attention.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.self_attn.k_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.self_attn.v_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.self_attn.q_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.self_attn.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.head.attention.out_proj:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encode_task_layer.0:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decode_task_layer.0:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decode_task_layer.2:
  Total Parameters: 1,328,256
  Trainable: 0
  Frozen: 1,328,256
  Trainable ratio: 0.00%

model.sana_vae.encoder.down_blocks.1.2.conv:
  Total Parameters: 1,180,160
  Trainable: 1,180,160
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.1.0.conv:
  Total Parameters: 1,179,904
  Trainable: 1,179,904
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.0.conv_out.conv_point:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.1.conv_out.conv_point:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.2.conv_out.conv_point:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.0.attn.to_q:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.0.attn.to_k:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.0.attn.to_v:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.1.attn.to_q:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.1.attn.to_k:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.1.attn.to_v:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.2.attn.to_q:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.2.attn.to_k:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.2.attn.to_v:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.0.attn.to_q:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.0.attn.to_k:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.0.attn.to_v:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.1.attn.to_q:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.1.attn.to_k:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.1.attn.to_v:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.2.attn.to_q:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.2.attn.to_k:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.2.attn.to_v:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.1.conv_out.conv_point:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.2.conv_out.conv_point:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.3.conv_out.conv_point:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.1.attn.to_q:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.1.attn.to_k:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.1.attn.to_v:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.2.attn.to_q:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.2.attn.to_k:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.2.attn.to_v:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.3.attn.to_q:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.3.attn.to_k:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.3.attn.to_v:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.0.attn.to_q:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.0.attn.to_k:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.0.attn.to_v:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.1.attn.to_q:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.1.attn.to_k:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.1.attn.to_v:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.2.attn.to_q:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.2.attn.to_k:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.2.attn.to_v:
  Total Parameters: 1,048,576
  Trainable: 1,048,576
  Frozen: 0
  Trainable ratio: 100.00%

model.vision_tower.vision_tower.encoder.embeddings.position_embedding:
  Total Parameters: 839,808
  Trainable: 0
  Frozen: 839,808
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.embeddings.patch_embedding:
  Total Parameters: 678,528
  Trainable: 0
  Frozen: 678,528
  Trainable ratio: 0.00%

model.sana_vae.encoder.down_blocks.1.0.conv1:
  Total Parameters: 590,080
  Trainable: 590,080
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.1.1.conv1:
  Total Parameters: 590,080
  Trainable: 590,080
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.1.1.conv1:
  Total Parameters: 590,080
  Trainable: 590,080
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.1.2.conv1:
  Total Parameters: 590,080
  Trainable: 590,080
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.1.3.conv1:
  Total Parameters: 590,080
  Trainable: 590,080
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.1.0.conv2:
  Total Parameters: 589,824
  Trainable: 589,824
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.1.1.conv2:
  Total Parameters: 589,824
  Trainable: 589,824
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.1.1.conv2:
  Total Parameters: 589,824
  Trainable: 589,824
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.1.2.conv2:
  Total Parameters: 589,824
  Trainable: 589,824
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.1.3.conv2:
  Total Parameters: 589,824
  Trainable: 589,824
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.time_embed.emb.timestep_embedder.linear_1:
  Total Parameters: 575,680
  Trainable: 575,680
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.0.attn.to_out:
  Total Parameters: 524,288
  Trainable: 524,288
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.1.attn.to_out:
  Total Parameters: 524,288
  Trainable: 524,288
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.2.attn.to_out:
  Total Parameters: 524,288
  Trainable: 524,288
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.1.attn.to_out:
  Total Parameters: 524,288
  Trainable: 524,288
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.2.attn.to_out:
  Total Parameters: 524,288
  Trainable: 524,288
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.3.attn.to_out:
  Total Parameters: 524,288
  Trainable: 524,288
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.conv_in:
  Total Parameters: 295,936
  Trainable: 295,936
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.0.2.conv:
  Total Parameters: 295,168
  Trainable: 295,168
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.0.0.conv:
  Total Parameters: 295,040
  Trainable: 295,040
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.conv_out:
  Total Parameters: 294,944
  Trainable: 294,944
  Frozen: 0
  Trainable ratio: 100.00%

model.vision_tower.vision_tower.decoder.vision_model.embeddings.position_embedding:
  Total Parameters: 294,912
  Trainable: 0
  Frozen: 294,912
  Trainable ratio: 0.00%

model.sana_vae.encoder.down_blocks.3.0.attn.to_q:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.0.attn.to_k:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.0.attn.to_v:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.1.attn.to_q:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.1.attn.to_k:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.1.attn.to_v:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.2.attn.to_q:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.2.attn.to_k:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.2.attn.to_v:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.1.attn.to_q:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.1.attn.to_k:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.1.attn.to_v:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.2.attn.to_q:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.2.attn.to_k:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.2.attn.to_v:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.3.attn.to_q:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.3.attn.to_k:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.3.attn.to_v:
  Total Parameters: 262,144
  Trainable: 262,144
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.0.0.conv1:
  Total Parameters: 147,584
  Trainable: 147,584
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.0.1.conv1:
  Total Parameters: 147,584
  Trainable: 147,584
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.0.1.conv1:
  Total Parameters: 147,584
  Trainable: 147,584
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.0.2.conv1:
  Total Parameters: 147,584
  Trainable: 147,584
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.0.3.conv1:
  Total Parameters: 147,584
  Trainable: 147,584
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.0.0.conv2:
  Total Parameters: 147,456
  Trainable: 147,456
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.0.1.conv2:
  Total Parameters: 147,456
  Trainable: 147,456
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.0.1.conv2:
  Total Parameters: 147,456
  Trainable: 147,456
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.0.2.conv2:
  Total Parameters: 147,456
  Trainable: 147,456
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.0.3.conv2:
  Total Parameters: 147,456
  Trainable: 147,456
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.0.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.1.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.2.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.3.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.4.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.5.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.6.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.7.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.8.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.9.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.10.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.11.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.12.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.13.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.14.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.15.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.16.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.17.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.18.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.19.ff.conv_depth:
  Total Parameters: 112,000
  Trainable: 112,000
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.0.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 98,304
  Trainable: 98,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.1.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 98,304
  Trainable: 98,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.2.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 98,304
  Trainable: 98,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.0.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 98,304
  Trainable: 98,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.1.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 98,304
  Trainable: 98,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.2.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 98,304
  Trainable: 98,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.1.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 98,304
  Trainable: 98,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.2.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 98,304
  Trainable: 98,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.3.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 98,304
  Trainable: 98,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.0.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 98,304
  Trainable: 98,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.1.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 98,304
  Trainable: 98,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.2.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 98,304
  Trainable: 98,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.0.conv_out.conv_depth:
  Total Parameters: 81,920
  Trainable: 81,920
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.1.conv_out.conv_depth:
  Total Parameters: 81,920
  Trainable: 81,920
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.2.conv_out.conv_depth:
  Total Parameters: 81,920
  Trainable: 81,920
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.0.conv_out.conv_depth:
  Total Parameters: 81,920
  Trainable: 81,920
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.1.conv_out.conv_depth:
  Total Parameters: 81,920
  Trainable: 81,920
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.2.conv_out.conv_depth:
  Total Parameters: 81,920
  Trainable: 81,920
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.1.conv_out.conv_depth:
  Total Parameters: 81,920
  Trainable: 81,920
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.2.conv_out.conv_depth:
  Total Parameters: 81,920
  Trainable: 81,920
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.3.conv_out.conv_depth:
  Total Parameters: 81,920
  Trainable: 81,920
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.0.conv_out.conv_depth:
  Total Parameters: 81,920
  Trainable: 81,920
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.1.conv_out.conv_depth:
  Total Parameters: 81,920
  Trainable: 81,920
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.2.conv_out.conv_depth:
  Total Parameters: 81,920
  Trainable: 81,920
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.0.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 76,800
  Trainable: 76,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.1.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 76,800
  Trainable: 76,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.2.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 76,800
  Trainable: 76,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.0.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 76,800
  Trainable: 76,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.1.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 76,800
  Trainable: 76,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.2.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 76,800
  Trainable: 76,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.1.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 76,800
  Trainable: 76,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.2.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 76,800
  Trainable: 76,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.3.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 76,800
  Trainable: 76,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.0.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 76,800
  Trainable: 76,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.1.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 76,800
  Trainable: 76,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.2.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 76,800
  Trainable: 76,800
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.patch_embed.proj:
  Total Parameters: 73,920
  Trainable: 73,920
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.proj_out:
  Total Parameters: 71,712
  Trainable: 71,712
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.0.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 49,152
  Trainable: 49,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.1.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 49,152
  Trainable: 49,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.2.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 49,152
  Trainable: 49,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.1.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 49,152
  Trainable: 49,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.2.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 49,152
  Trainable: 49,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.3.attn.to_qkv_multiscale.0.proj_out:
  Total Parameters: 49,152
  Trainable: 49,152
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.0.conv_out.conv_depth:
  Total Parameters: 40,960
  Trainable: 40,960
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.1.conv_out.conv_depth:
  Total Parameters: 40,960
  Trainable: 40,960
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.2.conv_out.conv_depth:
  Total Parameters: 40,960
  Trainable: 40,960
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.1.conv_out.conv_depth:
  Total Parameters: 40,960
  Trainable: 40,960
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.2.conv_out.conv_depth:
  Total Parameters: 40,960
  Trainable: 40,960
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.3.conv_out.conv_depth:
  Total Parameters: 40,960
  Trainable: 40,960
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.0.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 38,400
  Trainable: 38,400
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.1.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 38,400
  Trainable: 38,400
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.2.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 38,400
  Trainable: 38,400
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.1.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 38,400
  Trainable: 38,400
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.2.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 38,400
  Trainable: 38,400
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.3.attn.to_qkv_multiscale.0.proj_in:
  Total Parameters: 38,400
  Trainable: 38,400
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.conv_in:
  Total Parameters: 3,584
  Trainable: 3,584
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.conv_out:
  Total Parameters: 3,459
  Trainable: 3,459
  Frozen: 0
  Trainable ratio: 100.00%

model.vision_tower.vision_tower.encoder.encoder.layers.0.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.0.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.1.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.1.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.2.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.2.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.3.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.3.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.4.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.4.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.5.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.5.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.6.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.6.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.7.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.7.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.8.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.8.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.9.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.9.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.10.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.10.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.11.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.11.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.12.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.12.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.13.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.13.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.14.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.14.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.15.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.15.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.16.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.16.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.17.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.17.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.18.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.18.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.19.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.19.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.20.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.20.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.21.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.21.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.22.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.22.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.23.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.23.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.24.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.24.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.25.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.25.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.26.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.encoder.layers.26.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.post_layernorm:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.encoder.head.layernorm:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.layer_norm1:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.layer_norm2:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.post_layernorm:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.vision_tower.vision_tower.decoder.vision_model.head.layernorm:
  Total Parameters: 2,304
  Trainable: 0
  Frozen: 2,304
  Trainable ratio: 0.00%

model.diffusion_connector.3:
  Total Parameters: 2,304
  Trainable: 2,304
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.caption_norm:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.0.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.0.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.0.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.0.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.1.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.1.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.1.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.1.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.2.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.2.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.2.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.2.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.3.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.3.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.3.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.3.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.4.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.4.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.4.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.4.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.5.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.5.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.5.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.5.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.6.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.6.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.6.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.6.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.7.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.7.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.7.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.7.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.8.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.8.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.8.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.8.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.9.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.9.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.9.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.9.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.10.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.10.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.10.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.10.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.11.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.11.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.11.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.11.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.12.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.12.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.12.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.12.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.13.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.13.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.13.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.13.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.14.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.14.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.14.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.14.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.15.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.15.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.15.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.15.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.16.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.16.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.16.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.16.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.17.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.17.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.17.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.17.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.18.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.18.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.18.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.18.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.19.attn1.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.19.attn1.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.19.attn2.norm_q:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.sana.transformer_blocks.19.attn2.norm_k:
  Total Parameters: 2,240
  Trainable: 2,240
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.0.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.0.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.1.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.1.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.2.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.2.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.3.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.3.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.4.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.4.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.5.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.5.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.6.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.6.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.7.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.7.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.8.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.8.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.9.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.9.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.10.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.10.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.11.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.11.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.12.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.12.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.13.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.13.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.14.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.14.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.15.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.15.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.16.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.16.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.17.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.17.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.18.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.18.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.19.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.19.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.20.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.20.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.21.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.21.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.22.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.22.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.23.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.23.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.24.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.24.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.25.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.25.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.26.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.26.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.27.input_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.27.post_attention_layernorm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.norm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.0.attn.norm_out:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.0.conv_out.norm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.1.attn.norm_out:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.1.conv_out.norm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.2.attn.norm_out:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.4.2.conv_out.norm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.0.attn.norm_out:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.0.conv_out.norm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.1.attn.norm_out:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.1.conv_out.norm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.2.attn.norm_out:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.5.2.conv_out.norm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.1.attn.norm_out:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.1.conv_out.norm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.2.attn.norm_out:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.2.conv_out.norm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.3.attn.norm_out:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.4.3.conv_out.norm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.0.attn.norm_out:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.0.conv_out.norm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.1.attn.norm_out:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.1.conv_out.norm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.2.attn.norm_out:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.5.2.conv_out.norm:
  Total Parameters: 2,048
  Trainable: 2,048
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.2.0.norm:
  Total Parameters: 1,024
  Trainable: 1,024
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.2.1.norm:
  Total Parameters: 1,024
  Trainable: 1,024
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.0.attn.norm_out:
  Total Parameters: 1,024
  Trainable: 1,024
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.0.conv_out.norm:
  Total Parameters: 1,024
  Trainable: 1,024
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.1.attn.norm_out:
  Total Parameters: 1,024
  Trainable: 1,024
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.1.conv_out.norm:
  Total Parameters: 1,024
  Trainable: 1,024
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.2.attn.norm_out:
  Total Parameters: 1,024
  Trainable: 1,024
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.3.2.conv_out.norm:
  Total Parameters: 1,024
  Trainable: 1,024
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.2.1.norm:
  Total Parameters: 1,024
  Trainable: 1,024
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.2.2.norm:
  Total Parameters: 1,024
  Trainable: 1,024
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.2.3.norm:
  Total Parameters: 1,024
  Trainable: 1,024
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.1.attn.norm_out:
  Total Parameters: 1,024
  Trainable: 1,024
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.1.conv_out.norm:
  Total Parameters: 1,024
  Trainable: 1,024
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.2.attn.norm_out:
  Total Parameters: 1,024
  Trainable: 1,024
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.2.conv_out.norm:
  Total Parameters: 1,024
  Trainable: 1,024
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.3.attn.norm_out:
  Total Parameters: 1,024
  Trainable: 1,024
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.3.3.conv_out.norm:
  Total Parameters: 1,024
  Trainable: 1,024
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.1.0.norm:
  Total Parameters: 512
  Trainable: 512
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.1.1.norm:
  Total Parameters: 512
  Trainable: 512
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.1.1.norm:
  Total Parameters: 512
  Trainable: 512
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.1.2.norm:
  Total Parameters: 512
  Trainable: 512
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.1.3.norm:
  Total Parameters: 512
  Trainable: 512
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.0.0.norm:
  Total Parameters: 256
  Trainable: 256
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.encoder.down_blocks.0.1.norm:
  Total Parameters: 256
  Trainable: 256
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.0.1.norm:
  Total Parameters: 256
  Trainable: 256
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.0.2.norm:
  Total Parameters: 256
  Trainable: 256
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.up_blocks.0.3.norm:
  Total Parameters: 256
  Trainable: 256
  Frozen: 0
  Trainable ratio: 100.00%

model.sana_vae.decoder.norm_out:
  Total Parameters: 256
  Trainable: 256
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.0.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.0.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.1.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.1.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.2.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.2.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.3.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.3.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.4.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.4.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.5.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.5.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.6.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.6.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.7.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.7.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.8.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.8.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.9.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.9.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.10.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.10.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.11.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.11.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.12.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.12.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.13.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.13.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.14.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.14.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.15.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.15.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.16.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.16.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.17.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.17.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.18.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.18.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.19.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.19.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.20.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.20.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.21.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.21.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.22.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.22.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.23.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.23.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.24.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.24.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.25.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.25.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.26.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.26.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.27.self_attn.q_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%

model.layers.27.self_attn.k_norm:
  Total Parameters: 128
  Trainable: 128
  Frozen: 0
  Trainable ratio: 100.00%


5. Suggested Freezing Strategy
--------------------------------------------------
# Suggested freezing strategy based on module names:

# Suggested modules to freeze:
pipeline.model.model.layers.0.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.0.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.0.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.0.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.1.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.1.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.1.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.1.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.2.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.2.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.2.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.2.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.3.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.3.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.3.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.3.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.4.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.4.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.4.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.4.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.5.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.5.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.5.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.5.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.6.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.6.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.6.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.6.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.7.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.7.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.7.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.7.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.8.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.8.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.8.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.8.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.9.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.9.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.9.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.9.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.10.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.10.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.10.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.10.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.11.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.11.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.11.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.11.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.12.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.12.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.12.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.12.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.13.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.13.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.13.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.13.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.14.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.14.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.14.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.14.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.15.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.15.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.15.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.15.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.16.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.16.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.16.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.16.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.17.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.17.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.17.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.17.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.18.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.18.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.18.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.18.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.19.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.19.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.19.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.19.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.20.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.20.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.20.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.20.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.21.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.21.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.21.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.21.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.22.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.22.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.22.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.22.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.23.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.23.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.23.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.23.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.24.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.24.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.24.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.24.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.25.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.25.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.25.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.25.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.26.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.26.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.26.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.26.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.layers.27.self_attn.q_norm.requires_grad_(False)
pipeline.model.model.layers.27.self_attn.k_norm.requires_grad_(False)
pipeline.model.model.layers.27.input_layernorm.requires_grad_(False)
pipeline.model.model.layers.27.post_attention_layernorm.requires_grad_(False)
pipeline.model.model.norm.requires_grad_(False)
pipeline.model.model.vision_tower.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.embeddings.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.embeddings.patch_embedding.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.embeddings.position_embedding.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.0.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.0.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.0.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.0.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.0.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.0.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.0.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.0.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.0.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.0.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.0.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.0.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.1.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.1.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.1.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.1.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.1.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.1.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.1.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.1.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.1.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.1.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.1.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.2.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.2.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.2.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.2.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.2.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.2.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.2.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.2.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.2.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.2.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.2.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.3.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.3.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.3.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.3.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.3.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.3.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.3.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.3.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.3.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.3.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.3.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.3.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.4.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.4.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.4.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.4.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.4.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.4.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.4.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.4.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.4.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.4.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.4.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.4.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.5.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.5.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.5.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.5.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.5.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.5.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.5.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.5.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.5.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.5.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.5.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.5.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.6.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.6.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.6.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.6.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.6.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.6.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.6.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.6.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.6.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.6.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.6.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.6.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.7.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.7.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.7.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.7.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.7.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.7.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.7.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.7.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.7.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.7.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.7.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.7.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.8.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.8.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.8.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.8.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.8.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.8.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.8.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.8.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.8.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.8.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.8.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.8.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.9.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.9.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.9.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.9.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.9.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.9.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.9.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.9.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.9.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.9.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.9.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.9.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.10.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.10.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.10.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.10.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.10.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.10.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.10.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.10.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.10.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.10.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.10.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.10.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.11.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.11.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.11.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.11.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.11.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.11.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.11.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.11.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.11.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.11.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.11.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.11.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.12.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.12.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.12.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.12.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.12.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.12.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.12.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.12.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.12.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.12.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.12.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.12.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.13.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.13.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.13.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.13.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.13.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.13.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.13.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.13.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.13.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.13.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.13.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.13.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.14.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.14.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.14.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.14.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.14.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.14.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.14.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.14.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.14.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.14.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.14.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.14.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.15.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.15.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.15.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.15.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.15.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.15.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.15.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.15.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.15.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.15.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.15.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.15.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.16.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.16.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.16.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.16.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.16.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.16.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.16.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.16.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.16.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.16.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.16.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.16.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.17.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.17.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.17.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.17.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.17.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.17.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.17.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.17.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.17.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.17.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.17.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.17.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.18.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.18.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.18.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.18.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.18.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.18.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.18.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.18.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.18.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.18.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.18.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.18.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.19.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.19.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.19.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.19.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.19.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.19.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.19.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.19.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.19.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.19.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.19.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.19.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.20.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.20.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.20.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.20.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.20.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.20.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.20.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.20.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.20.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.20.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.20.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.20.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.21.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.21.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.21.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.21.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.21.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.21.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.21.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.21.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.21.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.21.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.21.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.21.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.22.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.22.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.22.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.22.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.22.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.22.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.22.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.22.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.22.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.22.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.22.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.22.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.23.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.23.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.23.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.23.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.23.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.23.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.23.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.23.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.23.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.23.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.23.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.23.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.24.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.24.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.24.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.24.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.24.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.24.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.24.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.24.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.24.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.24.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.24.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.24.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.25.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.25.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.25.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.25.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.25.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.25.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.25.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.25.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.25.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.25.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.25.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.25.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.26.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.26.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.26.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.26.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.26.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.26.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.26.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.26.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.26.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.26.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.26.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.encoder.layers.26.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.post_layernorm.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.head.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.head.attention.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.head.attention.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.head.layernorm.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.head.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.head.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.head.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encoder.head.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.embeddings.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.embeddings.patch_embedding.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.embeddings.position_embedding.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.0.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.1.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.layer_norm1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.self_attn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.self_attn.k_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.self_attn.v_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.self_attn.q_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.self_attn.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.layer_norm2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.encoder.layers.2.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.post_layernorm.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.head.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.head.attention.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.head.attention.out_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.head.layernorm.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.head.mlp.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.head.mlp.activation_fn.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.head.mlp.fc1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decoder.vision_model.head.mlp.fc2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encode_task_layer.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encode_task_layer.0.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.encode_task_layer.1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decode_task_layer.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decode_task_layer.0.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decode_task_layer.1.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.decode_task_layer.2.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.bottleneck.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.bottleneck.in_linear.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.bottleneck.out_linear.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.bottleneck.regularizer.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.bottleneck.regularizer.embedding.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.bottleneck.regularizer.embedding_proj.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.scale_layer.requires_grad_(False)
pipeline.model.model.vision_tower.vision_tower.image_resize.requires_grad_(False)
pipeline.model.model.sana.caption_norm.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.0.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.0.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.0.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.0.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.0.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.0.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.1.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.1.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.1.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.1.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.1.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.1.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.2.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.2.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.2.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.2.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.2.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.2.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.3.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.3.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.3.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.3.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.3.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.3.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.4.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.4.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.4.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.4.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.4.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.4.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.5.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.5.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.5.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.5.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.5.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.5.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.6.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.6.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.6.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.6.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.6.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.6.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.7.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.7.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.7.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.7.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.7.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.7.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.8.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.8.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.8.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.8.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.8.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.8.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.9.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.9.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.9.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.9.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.9.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.9.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.10.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.10.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.10.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.10.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.10.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.10.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.11.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.11.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.11.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.11.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.11.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.11.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.12.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.12.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.12.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.12.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.12.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.12.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.13.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.13.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.13.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.13.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.13.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.13.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.14.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.14.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.14.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.14.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.14.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.14.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.15.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.15.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.15.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.15.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.15.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.15.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.16.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.16.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.16.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.16.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.16.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.16.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.17.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.17.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.17.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.17.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.17.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.17.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.18.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.18.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.18.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.18.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.18.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.18.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.19.norm1.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.19.attn1.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.19.attn1.norm_k.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.19.norm2.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.19.attn2.norm_q.requires_grad_(False)
pipeline.model.model.sana.transformer_blocks.19.attn2.norm_k.requires_grad_(False)
pipeline.model.model.sana.norm_out.requires_grad_(False)
pipeline.model.model.sana.norm_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.conv_in.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.0.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.0.0.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.0.0.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.0.0.conv1.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.0.0.conv2.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.0.0.norm.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.0.1.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.0.1.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.0.1.conv1.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.0.1.conv2.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.0.1.norm.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.0.2.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.0.2.conv.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.1.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.1.0.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.1.0.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.1.0.conv1.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.1.0.conv2.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.1.0.norm.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.1.1.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.1.1.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.1.1.conv1.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.1.1.conv2.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.1.1.norm.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.1.2.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.1.2.conv.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.2.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.2.0.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.2.0.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.2.0.conv1.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.2.0.conv2.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.2.0.norm.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.2.1.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.2.1.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.2.1.conv1.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.2.1.conv2.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.2.1.norm.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.2.2.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.2.2.conv.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.attn.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.attn.to_q.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.attn.to_k.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.attn.to_v.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.attn.to_qkv_multiscale.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.attn.to_qkv_multiscale.0.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.attn.to_qkv_multiscale.0.proj_in.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.attn.to_qkv_multiscale.0.proj_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.attn.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.attn.to_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.conv_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.conv_out.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.conv_out.conv_inverted.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.conv_out.conv_depth.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.conv_out.conv_point.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.0.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.attn.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.attn.to_q.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.attn.to_k.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.attn.to_v.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.attn.to_qkv_multiscale.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.attn.to_qkv_multiscale.0.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.attn.to_qkv_multiscale.0.proj_in.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.attn.to_qkv_multiscale.0.proj_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.attn.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.attn.to_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.conv_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.conv_out.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.conv_out.conv_inverted.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.conv_out.conv_depth.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.conv_out.conv_point.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.1.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.attn.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.attn.to_q.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.attn.to_k.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.attn.to_v.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.attn.to_qkv_multiscale.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.attn.to_qkv_multiscale.0.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.attn.to_qkv_multiscale.0.proj_in.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.attn.to_qkv_multiscale.0.proj_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.attn.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.attn.to_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.conv_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.conv_out.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.conv_out.conv_inverted.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.conv_out.conv_depth.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.conv_out.conv_point.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.2.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.3.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.3.3.conv.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.attn.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.attn.to_q.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.attn.to_k.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.attn.to_v.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.attn.to_qkv_multiscale.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.attn.to_qkv_multiscale.0.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.attn.to_qkv_multiscale.0.proj_in.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.attn.to_qkv_multiscale.0.proj_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.attn.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.attn.to_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.conv_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.conv_out.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.conv_out.conv_inverted.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.conv_out.conv_depth.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.conv_out.conv_point.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.0.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.attn.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.attn.to_q.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.attn.to_k.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.attn.to_v.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.attn.to_qkv_multiscale.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.attn.to_qkv_multiscale.0.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.attn.to_qkv_multiscale.0.proj_in.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.attn.to_qkv_multiscale.0.proj_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.attn.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.attn.to_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.conv_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.conv_out.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.conv_out.conv_inverted.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.conv_out.conv_depth.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.conv_out.conv_point.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.1.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.attn.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.attn.to_q.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.attn.to_k.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.attn.to_v.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.attn.to_qkv_multiscale.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.attn.to_qkv_multiscale.0.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.attn.to_qkv_multiscale.0.proj_in.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.attn.to_qkv_multiscale.0.proj_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.attn.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.attn.to_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.conv_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.conv_out.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.conv_out.conv_inverted.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.conv_out.conv_depth.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.conv_out.conv_point.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.2.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.3.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.4.3.conv.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.attn.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.attn.to_q.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.attn.to_k.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.attn.to_v.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.attn.to_qkv_multiscale.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.attn.to_qkv_multiscale.0.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.attn.to_qkv_multiscale.0.proj_in.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.attn.to_qkv_multiscale.0.proj_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.attn.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.attn.to_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.conv_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.conv_out.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.conv_out.conv_inverted.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.conv_out.conv_depth.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.conv_out.conv_point.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.0.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.attn.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.attn.to_q.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.attn.to_k.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.attn.to_v.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.attn.to_qkv_multiscale.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.attn.to_qkv_multiscale.0.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.attn.to_qkv_multiscale.0.proj_in.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.attn.to_qkv_multiscale.0.proj_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.attn.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.attn.to_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.conv_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.conv_out.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.conv_out.conv_inverted.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.conv_out.conv_depth.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.conv_out.conv_point.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.1.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.attn.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.attn.to_q.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.attn.to_k.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.attn.to_v.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.attn.to_qkv_multiscale.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.attn.to_qkv_multiscale.0.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.attn.to_qkv_multiscale.0.proj_in.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.attn.to_qkv_multiscale.0.proj_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.attn.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.attn.to_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.conv_out.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.conv_out.nonlinearity.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.conv_out.conv_inverted.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.conv_out.conv_depth.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.conv_out.conv_point.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.down_blocks.5.2.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.encoder.conv_out.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.0.1.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.0.2.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.0.3.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.1.1.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.1.2.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.1.3.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.2.1.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.2.2.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.2.3.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.3.1.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.3.1.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.3.2.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.3.2.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.3.3.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.3.3.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.4.1.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.4.1.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.4.2.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.4.2.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.4.3.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.4.3.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.5.0.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.5.0.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.5.1.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.5.1.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.5.2.attn.norm_out.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.up_blocks.5.2.conv_out.norm.requires_grad_(False)
pipeline.model.model.sana_vae.decoder.norm_out.requires_grad_(False)


6. Training Suggestions
--------------------------------------------------
# Training suggestions based on model structure:

- Vision module detected, suggested to freeze pre-trained vision encoder
- VAE module detected, suggested to freeze VAE encoder and decoder

Suggested only to train specific adapter layers or new modules
Total parameter count is large (4,382,690,307), recommended to use parameter-efficient fine-tuning like LoRA