Hello.
We're now inside the Electrical and
Computer Engineering Building where we're surrounded by classrooms,
teaching labs, and office space.
During the spring semester,
the students in the College of Engineering hold an annual Engineering Open House.
Students form teams and build an engineering project to showcase what they have learned.
The atrium we're standing in is often filled with these showcases.
While some of these projects demonstrate basic engineering or physics phenomena,
for example, the non-Newtonian fluids are always popular.
In recent years, more projects have involved machine
learning either directly or indirectly,
as part of a hardware demonstration.
Given the nature of these projects,
the algorithms are often standard techniques
that are introduced in most machine learning classes.
This is true of our course as well.
And in this module, we introduce two additional fundamental algorithms,
Naive Bayes and Gaussian Processes.
Both of these techniques are rooted in probability theory and provide
a different view on using data to generate a predictive model.
Naive Bayes is an algorithm that is based on Bayes theorem,
hence the name, in which one computes
the probability of a given classification from a set of features.
Normally, one would need to worry about all of
the conditional probabilities between the different features.
However, the Naive part of the name refers to
the assumption that the occurrence of the different features are independent.
In most of the cases, this assumption is not strictly
valid if the algorithm often performs well,
especially in cases with large numbers of features such as text documents.
As a result, and because of the speed with which it can be calculated,
Naive Bayes is a fundamental algorithm that often serves as
a benchmark against which to compare more sophisticated techniques.
Gaussian Processes, on the other hand,
are used to build models to make continuous or regressive predictions from data.
In the Gaussian Process,
the data are assumed to be composed of random variables that are normally distributed.
The joint distribution of all of these Gaussian distributions forms the Gaussian Process.
This machine learning technique is a lazy learner,
like the k-nearest neighbor algorithm.
A prior for the data is specified,
and the data constrained the set of
possible Gaussian distributions to produce the predictive model.
This is a very versatile model and can generate probabilistic classifications,
making it powerful and efficient.
As the number of features becomes too large however,
say more than a few tens of features,
this technique can become inefficient.
Together with the nearest neighbor algorithm,
linear and logistic regression,
decision trees and support vector classifications,
these two algorithms form the fundamental set of machine learning techniques,
with which you should become familiar.
Knowing these algorithms, will enable you to tackle
most classification or regression tasks.
The readings and notebooks in this module will demonstrate how these algorithms work,
and how to apply them to different types of data. Good luck.