This page features training materials and preparation tips for the International Machine Learning Competition.

🎓 Preparation Tips for Participants

Below, you will find a set of tips designed to help you prepare for the International Machine Learning Competition. These recommendations are tailored to support your success in the competition and enhance your skills:

1. Know the Competition Format
   Start by understanding the structure and requirements of each round: the Qualification Round focuses on diverse topics across all areas of machine learning, the Pre-Final Round includes a supervised exam based on a recent research paper, and the Final Round tests broad, timed problem-solving under exam conditions. Reviewing past IMLC problems will help you grasp the diversity and difficulty level of each round.


2. Focus on the Core Topics
   The problems in IMLC come from a range of machine learning areas, including the mathematical fundamentals, optimization and generalization, deep learning, frontier models, applications, and trustworthy AI. Make sure you are comfortable with the fundamental concepts, algorithms, and key tools in these topics to build a solid foundation for tackling problems.


3. Train Your Problem-Solving Skills
   Work on improving your intuition, reasoning, and analytical thinking by solving machine learning problems from textbooks, past competitions, and the resources recommended below. Practice deriving algorithms by hand, reading loss curves, identifying assumptions, and recognizing failure modes. These skills will help you approach even the most challenging IMLC problems effectively.


4. Learn from Mistakes
   Reflecting on your mistakes and learning from them is an essential part of growth in any competition. First, try to solve the problems as far as possible. Then, compare them to a given solution and evaluate at which steps you made mistakes and correct them accordingly.


5. Use the Available Resources
   Take advantage of past IMLC problem sets, recommended textbooks, and online platforms to sharpen your skills. Additionally, the IMLC team is available to provide assistance and guidance; do not hesitate to contact us for support.


6. Prepare to Read a Research Paper (Pre-Final Round)
   The Pre-Final Round gives you a recent machine learning research paper ahead of a supervised exam. Practice reading and summarizing papers, focusing on extracting the key method, understanding the structure of the experiments, and connecting findings to broader machine learning concepts. Reading papers that present a method and evaluate it will help develop this skill.


7. Simulate Timed Problem-Solving (Final Exam)
   Time management is essential for the Pre-Final Round and even more the Final Exam. Practice solving problems within set time limits to develop a sense of pacing.


8. Collaborate and Learn from Others
   Join study groups, machine learning clubs, or connect with IMLC Ambassadors to discuss strategies and share insights. Collaboration can help you explore new problem-solving approaches and stay motivated throughout your preparation.


9. Enjoy the Learning Experience
   Keep in mind that IMLC prioritizes learning and expanding your knowledge while participating. Engage with each problem, and treat every challenge as an opportunity to deepen your understanding of how machine learning models work, learn, and fail.

☑ Syllabus

The following outlines the core machine learning areas covered in IMLC, along with key concepts and tools that are fundamental to each area. This syllabus is not comprehensive, but it covers much of the ground you will encounter. Most problems do not require highly specialized knowledge; instead, they test general machine learning understanding. Keep in mind that subsequent rounds build on the concepts of previous rounds, so the ideas below carry forward as the competition progresses.

• Fundamentals:
  The fundamentals establish the classical core that everything else in machine learning builds on. This includes understanding how models like support vector machines, logistic regression, decision trees, and kernels work, and how to derive them by hand. Mastery of these concepts is essential for understanding more advanced topics.
  • Linear Models: Linear regression, logistic regression, decision boundaries
  • Support Vector Machines: Maximum-margin classifiers, the kernel trick
  • Decision Trees: Splitting criteria, entropy, information gain
  • Loss Function: \(L(\hat{y}, y)\) (Measuring how wrong a prediction is)
  • Gradient: \(\nabla_\theta L(\theta)\) (Direction of steepest increase of the loss)


• Optimization & Generalization:
  This area covers how machines actually learn: gradient descent, loss landscapes, regularization, and why models that merely memorize the training data fail to generalize to new data. These concepts allow you to reason about why a model trains well or poorly.
  • Gradient Descent: \(\theta \leftarrow \theta - \eta \nabla_\theta L(\theta)\) (Updating parameters to reduce loss)
  • Bias-Variance Tradeoff: Balancing underfitting against overfitting
  • Regularization: \(L(\theta) + \lambda \lVert \theta \rVert^2\) (Penalizing complexity to improve generalization)
  • Cross-Validation: Estimating how a model performs on unseen data
  • Overfitting: When a model fits training data but fails on new data


• Deep Learning:
  Deep learning deals with the architectures built from layers of learned representations. Understanding backpropagation, convolutional networks, recurrent networks, and the transformer helps explain how modern models process data and learn from it.
  • Backpropagation: \(\frac{\partial L}{\partial \theta_i} = \frac{\partial L}{\partial y}\frac{\partial y}{\partial \theta_i}\) (Chain rule applied through a network)
  • Activation Functions: ReLU, sigmoid, softmax, and why non-linearity matters
  • Convolutional Networks: Learned filters for spatial data such as images
  • Recurrent Networks: Processing sequences by carrying a hidden state forward
  • Attention & Transformers: Weighing which parts of the input matter for each output


• Frontier Models:
  Frontier models covers the current research edge of the field: large language models, fine-tuning, retrieval-augmented generation, and the efficiency techniques that make large models deployable. Understanding scale, adaptation, and efficiency helps explain why today's largest models behave the way they do.
  • Large Language Models: Pretraining on large corpora, next-token prediction
  • Fine-Tuning: Adapting a pretrained model to a specific task or domain
  • Parameter-Efficient Fine-Tuning: Adapting a model by training a small set of extra parameters, such as LoRA and adapters
  • Retrieval-Augmented Generation: Combining a model with an external knowledge source
  • Quantization: Storing weights with fewer bits to reduce memory use
  • Distillation: Training a smaller model to mimic a larger one


• Applications:
  Applications is about where machine learning methods land in practice: computer vision, natural language processing, healthcare, recommender systems, and scientific discovery. Understanding how the same underlying methods translate into deployed systems helps connect theory to real-world impact.
  • Computer Vision: Image classification, object detection, segmentation
  • Natural Language Processing: Text classification, translation, question answering
  • Recommender Systems: Predicting what a user is likely to want next
  • Healthcare Applications: Diagnosis support, risk prediction, imaging
  • Evaluation Metrics: Precision, recall, accuracy, and when each one misleads
  • Class Imbalance: Why accuracy alone can be a poor measure of a model's quality


• Trustworthy AI:
  Trustworthy AI deals with how models fail, how they can be attacked, and what responsible deployment demands. Understanding robustness, fairness, and alignment helps explain why a model that performs well on paper can still cause harm in practice.
  • Adversarial Examples: Small, crafted input changes that fool a model
  • Robustness: How well a model holds up under distribution shift or attack
  • Fairness: Whether a model's errors are distributed unevenly across groups
  • Alignment: Whether a model's behavior matches the intentions of its designers
  • Interpretability: Understanding why a model produced a given output

Additionally, the Pre-Final Round gives you a recent research paper to study ahead of a supervised exam. The Final Exam can also include questions related to the previous problems (e.g., the research paper) from the Pre-Final Round and Qualification Round. Consider checking out this page to understand better how IMLC differs from other competition formats and what to expect:

• About IMLC ↗ (How IMLC differs from other competitions and what to expect.)

📖 Book Recommendations

Most introductory machine learning, statistics, and deep learning textbooks are useful and contain the information required to approach the problems. For reference, please have a look at the following list of recommended books:

• Machine Learning Foundations:
• Stuart Russell and Peter Norvig. Artificial Intelligence: A Modern Approach. Pearson.
• Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer.
• Kevin P. Murphy. Probabilistic Machine Learning: An Introduction. MIT Press.
• Aurélien Géron. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow. O'Reilly Media.
• Shai Shalev-Shwartz and Shai Ben-David. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press.


• Statistical Learning:
• Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani. An Introduction to Statistical Learning. Springer.
• Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning. Springer.


• Deep Learning:
• Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press.
• Christopher M. Bishop and Hugh Bishop. Deep Learning: Foundations and Concepts. Springer.
• Simon J.D. Prince. Understanding Deep Learning. MIT Press.
• François Chollet. Deep Learning with Python. Manning Publications.
• Aston Zhang, Zachary C. Lipton, Mu Li, and Alexander J. Smola. Dive into Deep Learning. Cambridge University Press.


• Mathematics for Machine Learning:
• Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong. Mathematics for Machine Learning. Cambridge University Press.
• Gilbert Strang. Introduction to Linear Algebra. Wellesley-Cambridge Press.
• Dimitri P. Bertsekas and John N. Tsitsiklis. Introduction to Probability. Athena Scientific.
• Larry Wasserman. All of Statistics: A Concise Course in Statistical Inference. Springer.
• Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press.


• Natural Language Processing, Vision, and Reinforcement Learning:
• Daniel Jurafsky and James H. Martin. Speech and Language Processing. Pearson.
• Richard Szeliski. Computer Vision: Algorithms and Applications. Springer.
• Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press.