Lecture 11

Introduction to Artificial Neural Networks

Important

Deadline: Assignment 2 must be submitted no later than October 20, 2025, at 11 PM. Please refer to the assignment description available on Brightspace.

Prepare

• Russell and Norvig (2020), pages 750–788

Watch 3Blue1Brown videos on neural networks

• But what is a Neural Network? (19 minutes)
• Gradient descent, how neural networks learn (21 minutes)
• What is backpropagation really doing? (14 minutes)
• Backpropagation calculus (10 minutes)

Narrative of PyTorch’s inception

• PyTorch (2024) Official PyTorch Documentary - Powering the AI Revolution (36 minutes)

Participate

• slides (PDF)

Practice

• TensorFlow Playground
  • Dataset Options: Users can choose from four types of datasets: circular, XOR, Gaussian, and spiral.
  • Feature Engineering: Enables the creation of new features to improve model performance.
  • Model Architecture: Allows customization of neural network architecture, including varying the number of layers and neurons per layer.
  • Hyperparameter Tuning: Provides options to adjust learning rate, activation functions, regularization techniques, and task specifications to observe their effects on model training.
  • Suggestion 1: For the Gaussian dataset, which is linearly separable, configure a network without hidden layers and a single output neuron using the sigmoid activation function. This setup effectively constructs a logistic regression model.
  • Suggestion 2: The circular dataset is not linearly separable using only the original features \(x_1\) and \(x_2\). However, by creating new features, \(x_1^2\) and \(x_2^2\), the problem becomes linearly separable in the transformed feature space. A network with no hidden layers and a single output node is sufficient for this task.

References

Russell, Stuart, and Peter Norvig. 2020. Artificial Intelligence: A Modern Approach. 4th ed. Pearson. http://aima.cs.berkeley.edu/.