SPML · Fall 2026 · Sharif University of Technology

Welcome to the public page for the course on Security and Privacy in Machine Learning (SPML). The main objectives of the course are to introduce students to the principles of security and privacy in machine learning. The students become familiar with the vulnerabilities of machine learning in the training and inference phases and the methods to improve the robustness and privacy of machine learning models.

Course Logistics

For external enquiries, emergencies, or personal matters that you don’t wish to put in a private post, you can email me at sadeghzadeh@sharif.edu.

Instructor

Course Staff

Course Pages

Main References

No required textbook. This field moves faster than textbooks do. Every reading is a research paper. The main references for the course are many research papers in top-tier conferences and journals in computer security (SP, CCS, USENIX Security, EuroSP) and machine learning (NeurIPS, ICLR, ICML, CVPR, ECCV).

Grading Policy

Course Policy

This course considers topics involving personal and public privacy and security. As part of this investigation we will cover technologies whose abuse may infringe on the rights of others. As an instructor, I rely on the ethical use of these technologies. Unethical use may include circumvention of existing security or privacy measurements for any purpose, or the dissemination, promotion, or exploitation of vulnerabilities of these services. Exceptions to these guidelines may occur in the process of reporting vulnerabilities through public and authoritative channels. Any activity outside the letter or spirit of these guidelines will be reported to the proper authorities and may result in dismissal from the class. When in doubt, please contact the instructor for advice. Do not undertake any action which could be perceived as technology misuse anywhere and/or under any circumstances unless you have received explicit permission from Dr. Sadeghzadeh.

Academic Honesty

Homework Submission

Submit your answers in a .pdf or .zip file on the course page on the Quera website, with the following format: HW[HW#]-[FamilyName]-[std#] (for example, HW3-Hoseini-401234567).

Homework Policy

Late Policy

Schedule

Date	Day	Topic	Lecture	Reading
Oct 25 (3 Aban)	Sunday	Course Intro & Overview	Course Scope, Principles & Motivation	Towards the Science of Security and Privacy in ML (Papernot et al.) Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems (Dalrymple et al., 2024)
Oct 27 (5 Aban)	Tuesday	Adversarial Examples Intro	Vulnerabilities of Deep Learning & Threat Models	Intriguing Properties of Neural Networks (Szegedy et al.) Explaining and Harnessing Adversarial Examples (Goodfellow et al.)
Nov 01 (10 Aban)	Sunday	Iterative Adversarial Attacks	Projected Gradient Descent (PGD) & Carlini-Wagner (C&W) Attack Formulations	Towards Deep Learning Models Resistant to Adversarial Attacks (Madry et al.) Towards Evaluating the Robustness of Neural Networks (Carlini & Wagner)
Nov 03 (12 Aban)	Tuesday	Physical & Universal Attacks	Universal Adversarial Perturbations & Adversarial Patches	Universal Adversarial Perturbations; Adversarial Patch (Brown et al.)
Nov 08 (17 Aban)	Sunday	Black-Box Adversarial Attacks	Transferability, Query-Based Attacks & Zeroth-Order Optimization	Practical Black-Box Attacks; ZOO (Chen et al.) Black-box Adversarial Attacks with Limited Queries and Information
Nov 10 (19 Aban)	Tuesday	Obfuscated Gradients & Empirical Defenses Against AEs	Gradient Masking & Rigorous Robustness Evaluation	Obfuscated Gradients Give a False Sense of Security (Athalye et al.) Towards Deep Learning Models Resistant to Adversarial Attacks
Nov 15 (24 Aban)	Sunday	Certified Robustness	Randomized Smoothing & Convex Outer Adversarial Polytope	Certified Adversarial Robustness via Randomized Smoothing (Cohen et al.)
Nov 17 (26 Aban)	Tuesday	Data Poisoning Attacks	Targeted Clean-Label Poisoning & Bilevel Optimization	BadNets (Gu et al.); Poison Frogs! Targeted Clean-Label Poisoning Attacks (Shafahi et al.) Deep Partition Aggregation (Levine & Feizi) Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning Poisoning Web-Scale Training Datasets is Practical (Carlini et al., 2023)
Nov 22 (1 Azar)	Sunday	Model Extraction Attacks & Model IP & Watermarking	Model Fingerprinting, Watermarking & Verification	High Accuracy and High Fidelity Extraction; Knockoff Nets (Orekondy et al.) Turning Weakness Into Strength: Watermarking DNNs by Backdooring Imitation Attacks and Defenses for Black-box Machine Translation Systems (Wallace et al., 2020) Stealing Part of a Production Language Model (Carlini et al., 2024) Scalable watermarking for identifying large language model outputs (Dathathri et al., 2024)
Nov 24 (3 Azar)	Tuesday	Privacy Risks in ML	Membership Inference & Attribute Inference Attacks	Membership Inference Attacks against Machine Learning Models (Shokri et al.) Text Embeddings Reveal (Almost) As Much As Text (Morris et al., 2023) Deep Leakage from Gradients (Zhu et al., 2019)
Nov 29 (8 Azar)	Sunday	Federated Learning & Gradient Leakage	Membership Inference & Attribute Inference Attacks	Membership Inference Attacks From First Principles (Carlini et al., 2021) Do Membership Inference Attacks Work on Large Language Models? (Duan et al., 2024)
Dec 01 (10 Azar)	Tuesday	Differential Privacy Foundations	(ε, δ)-Differential Privacy & Randomized Response	The Algorithmic Foundations of Differential Privacy (Dwork & Roth)
Dec 06 (15 Azar)	Sunday	Differential Privacy Mechanisms	Laplace, Gaussian Mechanisms & Composition Theorems	The Algorithmic Foundations of Differential Privacy; Renyi DP (Mironov)
Dec 08 (17 Azar)	Tuesday	Private Deep Learning	DP-SGD & PATE (Private Aggregation of Teacher Ensembles)	Deep Learning with DP (Abadi et al.); Semi-supervised Transfer (Papernot et al.)
Dec 13 (22 Azar)	Sunday	Introduction to Large Language models and Alignment	Autoregressive Language Modeling, Pretraining, In-Context Learning & Alignment Objectives	Language Models are Few-Shot Learners (Brown et al., 2020) Training language models to follow instructions with human feedback (Ouyang et al., 2022) Constitutional AI: Harmlessness from AI Feedback (Bai et al., 2022)
Dec 15 (24 Azar)	Tuesday	RLHF	Supervised Fine-Tuning, Preference-Based Reward Modeling & KL-Regularized Policy Optimization	Fine-Tuning Language Models from Human Preferences (Ziegler et al., 2019) Training language models to follow instructions with human feedback (Ouyang et al., 2022) Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Dec 20 (29 Azar)	Sunday	LLM safety alignment and jailbreaking	Safety Training, Refusal Behavior & Transferable Adversarial Prompts	Jailbroken: How Does LLM Safety Training Fail? (Wei et al., 2023) Universal and Transferable Adversarial Attacks on Aligned Language Models (Zou et al., 2023) Recommended: Are aligned neural networks adversarially aligned? (Carlini et al., 2023) AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
Dec 22 (1 Dey)	Tuesday	LLM Red-teaming	Automated Attack Search, Adaptive Multi-Turn Jailbreaks & Red-Teaming Evaluation	Tree of Attacks: Jailbreaking Black-Box LLMs Automatically X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents
Dec 27 (6 Dey)	Sunday	Agentic AI	Reasoning–Action Loops, Tool Use, Planning, Memory & Verbal Feedback	ReAct: Synergizing Reasoning and Acting in Language Models Toolformer: LMs Can Teach Themselves to Use Tools Generative Agents: Interactive Simulacra of Human Behavior Tree of Thoughts: Deliberate Problem Solving with LLMs
Dec 29 (8 Dey)	Tuesday	Threat Modeling	Agent Assets, Trust Boundaries, Attacker Capabilities & Security Evaluation	Agent Security Bench (ASB) Identifying the Risks of LM Agents with an LM-Emulated Sandbox (ToolEmu) -- AgentHarm: Measuring Harmfulness of LLM Agents Attacking Vision-Language Computer Agents via Pop-ups
Jan 03 (13 Dey)	Sunday	Prompt Injection I	Direct & Indirect Prompt Injection, Instruction–Data Confusion & Attack Surfaces	Not What You've Signed Up For: Indirect Prompt Injection Formalizing and Benchmarking Prompt Injection Attacks and Defenses
Jan 05 (15 Dey)	Tuesday	Prompt Injection II	Tool-Integrated Injection Attacks, Adaptive Adversaries & Benchmark-Based Evaluation	InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Agents AgentDojo: A Dynamic Environment to Evaluate Attacks and Defenses
Jan 10 (20 Dey)	Sunday	Poisoning Memory, State, and Persistent Attacks	Memory & Knowledge-Base Poisoning, Agent Backdoors & Cross-Session Persistence	AgentPoison: Red-Teaming LLM Agents via Poisoning Memory or Knowledge Bases Watch Out for Your Agents! Backdoor Threats to LLM-Based Agents Poisoning Language Models During Instruction Tuning
Jan 12 (22 Dey)	Tuesday	Tools, Privileges, and Authorization	Tool-Interface Threats, Least Privilege, Authorization Boundaries & Capability-Based Controls	Model Context Protocol (MCP): Landscape and Security Threats Defeating Prompt Injections by Design (CaMeL)
Jan 17 (27 Dey)	Sunday	Multi-Agent Security	Attack Propagation, Compromised Agents & Resilient Multi-Agent Collaboration	Agent Smith: A Single Image Can Jailbreak One Million Multimodal Agents On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents
Jan 19 (29 Dey)	Tuesday	Defenses I: Architecture	Execution Isolation, Trust Separation & Communication Firewalls	The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems Firewalls to Secure Dynamic LLM Agentic Networks
Jan 26 (6 Bahman)	Tuesday	Defenses II: Model and Runtime	Structured Queries, Security Preference Optimization & Runtime Enforcement	StruQ: Defending Against Prompt Injection with Structured Queries SecAlign: Defending Against Prompt Injection with Preference Optimization
Jan 31 (11 Bahman)	Sunday	Unlearning	Unlearning (and why it's hard)	Machine Unlearning Machine Unlearning in 2024 (Liu) Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy, Research, and Practice (A. Feder Cooper et al., 2024)

Security and Privacy in Machine Learning