Hugging Face: Pretrained Models in Practice
The Hub, pipelines, tokenizers, Auto classes, the datasets library, fine-tuning with Trainer, LoRA with peft, and reading model cards and licences.
The ecosystem
| Piece | What it does |
|---|---|
| Hub (huggingface.co) | hosts hundreds of thousands of models and datasets, each with a model card (training data, intended use, limitations, licence) |
transformers | load and run pretrained models: pipeline, AutoTokenizer, AutoModelFor…, Trainer, generate |
datasets | load, split and process datasets efficiently (load_dataset, map) |
peft | parameter-efficient fine-tuning such as LoRA |
accelerate, evaluate, sentence-transformers | multi-GPU and mixed precision; metrics; sentence embeddings |
The models are PyTorch models underneath: everything from PyTorch Essentials applies.
Pipelines: a working model in one line
from transformers import pipeline
clf = pipeline("text-classification", model="distilbert/distilbert-base-uncased-finetuned-sst-2-english")
clf("The clinic staff were wonderful.") # [{'label': 'POSITIVE', 'score': 0.9998}]
pipeline("zero-shot-classification", model="facebook/bart-large-mnli")("Can I move my booking?", candidate_labels=["booking", "billing"])
pipeline("summarization"), pipeline("ner", aggregation_strategy="simple"), pipeline("automatic-speech-recognition")
Always pass an explicit model= (the default can change between versions) and check the model card before relying on it.
Try it: which class and model for my task?
Choose a task to see the pipeline name, the AutoModelFor… class for fine-tuning, and typical starting models.
Interactive: this part runs in your browser.
Tokenizers and Auto classes
from transformers import AutoTokenizer, AutoModelForSequenceClassification name = "distilbert-base-uncased" tok = AutoTokenizer.from_pretrained(name) # always the tokenizer that matches the model model = AutoModelForSequenceClassification.from_pretrained(name, num_labels=3) # new head, random init batch = tok(["first text", "a second, longer text"], padding=True, truncation=True, max_length=128, return_tensors="pt") out = model(**batch) # out.logits: (2, 3)
AutoModel gives hidden states; AutoModelFor<Task> adds a task head (classification, token classification, question answering, causal LM…). A warning that some weights were "newly initialized" is expected for a new head: it must be fine-tuned.
Code example: what the tokenizer gives your model
A worked example: read it step by step (every part is explained), then run it and experiment.
Interactive: this part runs in your browser.
datasets, Trainer and LoRA
from datasets import load_dataset
from transformers import TrainingArguments, Trainer, DataCollatorWithPadding
ds = load_dataset("imdb") # or load_dataset("csv", data_files=...)
tok_ds = ds.map(lambda b: tok(b["text"], truncation=True, max_length=256), batched=True)
args = TrainingArguments("out", learning_rate=2e-5, per_device_train_batch_size=16, num_train_epochs=2,
eval_strategy="epoch", fp16=True, report_to="none")
trainer = Trainer(model=model, args=args, train_dataset=tok_ds["train"], eval_dataset=tok_ds["test"],
data_collator=DataCollatorWithPadding(tok), compute_metrics=compute_metrics)
trainer.train(); trainer.save_model("my-model")
from peft import LoraConfig, get_peft_model # train ~1% of parameters
model = get_peft_model(model, LoraConfig(task_type="SEQ_CLS", r=8, lora_alpha=16, target_modules=["q_lin", "v_lin"]))
Fine-tuning learning rates are small (about 1e-5 to 5e-5 for full fine-tuning, about 1e-4 to 1e-3 for LoRA). Argument names change between versions (for example evaluation_strategy became eval_strategy), so check the docs for your installed version.
Try it: how many parameters does LoRA train?
Change the rank and which matrices get adapters, and compare the trainable parameters with full fine-tuning.
Interactive: this part runs in your browser.
Model cards, licences and safety
- Read the model card: training data, intended use, evaluation, known limitations and biases.
- Check the licence (Apache-2.0, MIT, Llama or Gemma licences, CC-BY-NC…): some forbid commercial use or require attribution.
- Prefer safetensors weight files, and be careful with
trust_remote_code=True: it runs code from the repository on your machine. - Gated models need you to accept terms and log in with an access token (
huggingface-cli login); never put tokens in code you share. - Pin the model revision (a commit hash) in production so the model cannot change under you.
Hands-on: Hugging Face in Colab (notebook 14)
Pipelines, tokenizers, sentence embeddings with masked mean pooling, datasets, fine-tuning DistilBERT with Trainer, LoRA with peft, and saving and reloading.
Interactive: this part runs in your browser.
Cheat sheet: Hugging Face
The commands you will use most, on one screen. Come back to it while you work.
Interactive: this part runs in your browser.
Quiz
1. Why pass an explicit model name to pipeline()?
2. "Some weights were newly initialized" when loading AutoModelForSequenceClassification means…
3. What does the attention mask tell the model?
4. A typical learning rate for full fine-tuning of a BERT-size model is about…
5. trust_remote_code=True is risky because…
📚 Further reading
Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.
- Hugging Face LLM Course ↗ Free course on transformers and LLMs
- Hugging Face Transformers documentation ↗ pipelines, Auto classes, Trainer
- Hugging Face Datasets documentation ↗ loading and processing datasets
- Hugging Face PEFT documentation ↗ LoRA and other parameter-efficient methods