About this Course
34,063

100% online

Start instantly and learn at your own schedule.

Flexible deadlines

Reset deadlines in accordance to your schedule.

Advanced Level

This is an advanced course, intended for learners with a background in computer vision and deep learning.

Approx. 20 hours to complete

Suggested: 6 weeks of study, 5-6 hours per week...

English

Subtitles: English

What you will learn

  • Check

    Work with the pinhole camera model, and perform intrinsic and extrinsic camera calibration

  • Check

    Detect, describe and match image features and design your own convolutional neural networks

  • Check

    Apply these methods to visual odometry, object detection and tracking

  • Check

    Apply semantic segmentation for drivable surface estimation

100% online

Start instantly and learn at your own schedule.

Flexible deadlines

Reset deadlines in accordance to your schedule.

Advanced Level

This is an advanced course, intended for learners with a background in computer vision and deep learning.

Approx. 20 hours to complete

Suggested: 6 weeks of study, 5-6 hours per week...

English

Subtitles: English

Syllabus - What you will learn from this course

Week
1
2 hours to complete

Welcome to Course 3: Visual Perception for Self-Driving Cars

This module introduces the main concepts from the broad field of computer vision needed to progress through perception methods for self-driving vehicles. The main components include camera models and their calibration, monocular and stereo vision, projective geometry, and convolution operations....
4 videos (Total 18 min), 4 readings
4 videos
Welcome to the course4m
Meet the Instructor, Steven Waslander5m
Meet the Instructor, Jonathan Kelly2m
4 readings
Course Prerequisites15m
How to Use Discussion Forums15m
How to Use Supplementary Readings in This Course15m
Recommended Textbooks15m
7 hours to complete

Module 1: Basics of 3D Computer Vision

This module introduces the main concepts from the broad field of computer vision needed to progress through perception methods for self-driving vehicles. The main components include camera models and their calibration, monocular and stereo vision, projective geometry, and convolution operations....
6 videos (Total 43 min), 4 readings, 2 quizzes
6 videos
Lesson 1 Part 2: Camera Projective Geometry8m
Lesson 2: Camera Calibration7m
Lesson 3 Part 1: Visual Depth Perception - Stereopsis7m
Lesson 3 Part 2: Visual Depth Perception - Computing the Disparity5m
Lesson 4: Image Filtering7m
4 readings
Supplementary Reading: The Camera Sensor30m
Supplementary Reading: Camera Calibration15m
Supplementary Reading: Visual Depth Perception30m
Supplementary Reading: Image Filtering15m
1 practice exercise
Module 1 Graded Quiz30m
Week
2
7 hours to complete

Module 2: Visual Features - Detection, Description and Matching

Visual features are used to track motion through an environment and to recognize places in a map. This module describes how features can be detected and tracked through a sequence of images and fused with other sources for localization as described in Course 2. Feature extraction is also fundamental to object detection and semantic segmentation in deep networks, and this module introduces some of the feature detection methods employed in that context as well....
6 videos (Total 44 min), 5 readings, 1 quiz
6 videos
Lesson 2: Feature Descriptors6m
Lesson 3 Part 1: Feature Matching7m
Lesson 3 Part 2: Feature Matching: Handling Ambiguity in Matching5m
Lesson 4: Outlier Rejection8m
Lesson 5: Visual Odometry9m
5 readings
Supplementary Reading: Feature Detectors and Descriptors30m
Supplementary Reading: Feature Matching15m
Supplementary Reading: Feature Matching15m
Supplementary Reading: Outlier Rejection15m
Supplementary Reading: Visual Odometry10m
Week
3
3 hours to complete

Module 3: Feedforward Neural Networks

Deep learning is a core enabling technology for self-driving perception. This module briefly introduces the core concepts employed in modern convolutional neural networks, with an emphasis on methods that have been proven to be effective for tasks such as object detection and semantic segmentation. Basic network architectures, common components and helpful tools for constructing and training networks are described....
6 videos (Total 58 min), 6 readings, 1 quiz
6 videos
Lesson 2: Output Layers and Loss Functions10m
Lesson 3: Neural Network Training with Gradient Descent10m
Lesson 4: Data Splits and Neural Network Performance Evaluation8m
Lesson 5: Neural Network Regularization9m
Lesson 6: Convolutional Neural Networks9m
6 readings
Supplementary Reading: Feed-Forward Neural Networks15m
Supplementary Reading: Output Layers and Loss Functions15m
Supplementary Reading: Neural Network Training with Gradient Descent15m
Supplementary Reading: Data Splits and Neural Network Performance Evaluation10m
Supplementary Reading: Neural Network Regularization15m
Supplementary Reading: Convolutional Neural Networks10m
1 practice exercise
Feed-Forward Neural Networks30m
Week
4
3 hours to complete

Module 4: 2D Object Detection

The two most prevalent applications of deep neural networks to self-driving are object detection, including pedestrian, cyclists and vehicles, and semantic segmentation, which associates image pixels with useful labels such as sign, light, curb, road, vehicle etc. This module presents baseline techniques for object detection and the following module introduce semantic segmentation, both of which can be used to create a complete self-driving car perception pipeline....
4 videos (Total 52 min), 4 readings, 1 quiz
4 videos
Lesson 2: 2D Object detection with Convolutional Neural Networks11m
Lesson 3: Training vs. Inference11m
Lesson 4: Using 2D Object Detectors for Self-Driving Cars14m
4 readings
Supplementary Reading: The Object Detection Problem15m
Supplementary Reading: 2D Object detection with Convolutional Neural Networks30m
Supplementary Reading: Training vs. Inference45m
Supplementary Reading: Using 2D Object Detectors for Self-Driving Cars30m
1 practice exercise
Object Detection For Self-Driving Cars30m
Week
5
2 hours to complete

Module 5: Semantic Segmentation

The second most prevalent application of deep neural networks to self-driving is semantic segmentation, which associates image pixels with useful labels such as sign, light, curb, road, vehicle etc. The main use for segmentation is to identify the drivable surface, which aids in ground plane estimation, object detection and lane boundary assessment. Segmentation labels are also being directly integrated into object detection as pixel masks, for static objects such as signs, lights and lanes, and moving objects such cars, trucks, bicycles and pedestrians. ...
3 videos (Total 31 min), 3 readings, 1 quiz
3 videos
Lesson 2: ConvNets for Semantic Segmentation11m
Lesson 3: Semantic Segmentation for Road Scene Understanding11m
3 readings
Supplementary Reading: The Semantic Segmentation Problem30m
Supplementary Reading: ConvNets for Semantic Segmentation30m
Supplementary Reading: Semantic Segmentation for Road Scene Understanding30m
1 practice exercise
Semantic Segmentation For Self-Driving Cars20m
Week
6
7 hours to complete

Module 6: Putting it together - Perception of dynamic objects in the drivable region

The final module of this course focuses on the implementation of a collision warning system that alerts a self-driving car about the position and category of obstacles present in their lane. The project is comprised of three major segments: 1) Estimating the drivable space in 3D, 2) Semantic Lane Estimation and 3) Filter wrong output from object detection using semantic segmentation....
4 videos (Total 24 min), 1 quiz
4 videos
Final Project Hints6m
Final Project Solution [LOCKED]9m
Congratulations for completing the course!3m

Instructor

Avatar

Steven Waslander

Associate Professor
Aerospace Studies

About University of Toronto

Established in 1827, the University of Toronto is one of the world’s leading universities, renowned for its excellence in teaching, research, innovation and entrepreneurship, as well as its impact on economic prosperity and social well-being around the globe. ...

About the Self-Driving Cars Specialization

Be at the forefront of the autonomous driving industry. With market researchers predicting a $42-billion market and more than 20 million self-driving cars on the road by 2025, the next big job boom is right around the corner. This Specialization gives you a comprehensive understanding of state-of-the-art engineering practices used in the self-driving car industry. You'll get to interact with real data sets from an autonomous vehicle (AV)―all through hands-on projects using the open source simulator CARLA. Throughout your courses, you’ll hear from industry experts who work at companies like Oxbotica and Zoox as they share insights about autonomous technology and how that is powering job growth within the field. You’ll learn from a highly realistic driving environment that features 3D pedestrian modelling and environmental conditions. When you complete the Specialization successfully, you’ll be able to build your own self-driving software stack and be ready to apply for jobs in the autonomous vehicle industry. It is recommended that you have some background in linear algebra, probability, statistics, calculus, physics, control theory, and Python programming. You will need these specifications in order to effectively run the CARLA simulator: Windows 7 64-bit (or later) or Ubuntu 16.04 (or later), Quad-core Intel or AMD processor (2.5 GHz or faster), NVIDIA GeForce 470 GTX or AMD Radeon 6870 HD series card or higher, 8 GB RAM, and OpenGL 3 or greater (for Linux computers)....
Self-Driving Cars

Frequently Asked Questions

  • Once you enroll for a Certificate, you’ll have access to all videos, quizzes, and programming assignments (if applicable). Peer review assignments can only be submitted and reviewed once your session has begun. If you choose to explore the course without purchasing, you may not be able to access certain assignments.

  • When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile. If you only want to read and view the course content, you can audit the course for free.

More questions? Visit the Learner Help Center.