The math behind a prediction

Begin with something you can see

Three observations sit on a graph. A line offers a simple prediction: multiply the input by a slope. Move the slope and notice how far the predictions fall from the observations.

The dashed vertical gaps are residuals: observed value minus prediction. Squaring each gap and taking the average gives the displayed error. This tiny example is a teaching addition, not a model taught in the original notes.

For inputs $\mathbf x=(1,2,3)^\top$ and observations $\mathbf y=(2,3,5)^\top$, the predictions are $w\mathbf x$. The average squared error is

Linear algebra organizes the three numbers. Calculus tells us how a small change to $w$ changes $L$. Probability helps us describe why observed values might vary even at the same input. You do not need to solve the optimization yet. First recognize the different jobs.

What the original notes promise

Garrett Thomas wrote Mathematics for Machine Learning at the Department of Electrical Engineering and Computer Sciences, University of California, Berkeley. The supplied version is dated January 11, 2018.

Its purpose is to summarize the mathematical background for an introductory machine learning class, called CS 189/289A at UC Berkeley. Machine learning draws on several mathematical fields; these notes collect some of their essential tools.

The source assumes familiarity with multivariable calculus and linear algebra at the level of Berkeley Math 53/54. It explicitly says it is not a replacement for those prerequisite classes. Most topics are treated briefly. References direct interested readers to fuller treatments.

That matters for this website: module 0 supplies an introductory bridge, while later lessons unpack the source in more detail. The bridge is a starting point, not the equivalent of two complete college mathematics courses.

Mathematics for ML, rather than a catalog of algorithms

The source is about mathematical background, not machine learning itself. It does not teach specific models or algorithms except for occasional mentions that show why a mathematical idea matters. This website adds small applications to make the math tangible. They are labeled as teaching examples; they do not change the original scope.

Earlier versions of the source had no proofs. The author began adding proofs when they were reasonably short and helped understanding. Those proofs were not required background for CS 189, but were included to deepen understanding. Here, all proofs supplied by the source will remain available when their sections are implemented.

How to use this course

1. Observe: describe what changes in a figure before reading the formula.
2. Predict: decide what you expect a control to do.
3. Explain: connect the numerical change to the symbols.
4. Generalize: read the definition, assumptions, and proof.
5. Check: try a case where the assumptions fail.

If symbols are unfamiliar, continue through this module. If functions or calculus feel uncertain, start with module 0. Neither route requires memorizing the whole symbol table in advance.

Source access and corrections

The author states that the document may be freely distributed. The source gives this location for its latest version, and asks that mistakes be reported to gwthomas@berkeley.edu. This project follows the local 47-page PDF, so a later online version may differ. These links reproduce the source’s access and correction information; the website does not send any message on your behalf.