SFTTrainer Source Code Exploration: Prepare Train

• Prepare Train Overall Logic
• Prepare Train Code Details
  • _inner_training_loop
  • training_step
  • compute_loss
  • PeftModelForCausalLM.forward
  • Linear4bit.forward

SFTTrainer Source Code Exploration: Prepare Dataset

• Prepare Dataset Overall Logic
• Prepare Dataset Code Details
  • SFTTrainer.init
  • DataCollatorForLanguageModeling
  • _prepare_dataset

SFTTrainer Source Code Exploration: Prepare Model

• Prepare Model Overall Logic
• Prepare Model Code Details
  • _prepare_peft_model
  • PeftModelForCausalLM.init
  • PeftModel.init
  • LoraModel.init
  • Linear4bit.init
  • LoraLayer.init(self, base_layer)