Introduction to Artificial Neural Networks

CSI 4106 - Fall 2024

Marcel Turcotte

Version: Nov 14, 2024 09:02

Preamble

Quote of the Day

The Nobel Prize in Physics 2024 was awarded to John J. Hopfield and Geoffrey E. Hinton “for foundational discoveries and inventions that enable machine learning with artificial neural networks”

Learning objectives

• Explain perceptrons and MLPs: structure, function, history, and limitations.
• Describe activation functions: their role in enabling complex pattern learning.
• Implement a feedforward neural network with Keras on Fashion-MNIST.
• Interpret neural network training and results: visualization and evaluation metrics.
• Familiarize with deep learning frameworks: PyTorch, TensorFlow, and Keras for model building and deployment.

As stated at the beginning of this course, there are two primary schools of thought in artificial intelligence: symbolic AI and connectionism. While the symbolic approach initially dominated the field, the connectionist approach is now more prevalent. We will now focus on connectionism.

Introduction

Neural Networks (NN)

We now shift our focus to a family of machine learning models that draw inspiration from the structure and function of biological neural networks found in animals.

AKA artificial neural networks or neural nets, abbreviated as ANN or NN.

Machine Learning Problems

• Supervised Learning: Classification, Regression

• Unsupervised Learning: Autoencoders, Self-Supervised

• Reinforcement Learning: Now an Integral Component

We will begin our exploration within the framework of supervised learning.

A neuron

In the study of artificial intelligence, it is logical to derive inspiration from the most well-understood form of intelligence: the human brain. The brain is composed of a complex network of neurons, which together form biological neural networks. Although each neuron exhibits relatively simple behavior, it is connected to thousands of other neurons, contributing to the intricate functionality of these networks.

A neuron can be conceptualized as a basic computational unit, and the complexity of brain function arises from the interconnectedness of these units.

Yann LeCun and other researchers have frequently noted that artificial neural networks used in machine learning resemble biological neural networks in much the same way that an airplane’s wings resemble those of a bird.

Attribution: Jennifer Walinga, CC BY-SA 4.0

Interconnected neurons

From biology, we essentially adopt the concept of simple computational units that are interconnected to form a network, which collectively performs complex computations.

While research into understanding biological neural networks is undeniably important, the field of artificial neural networks has incorporated only a limited number of key concepts from this research.

Attribution: Molecular Mechanism of Synaptic Function from the Howard Hughes Medical Institute (HHMI). Published on YouTube on 2018-11-15.

Connectionist

Another characteristic of biological neural networks that we adopt is the organization of neurons into layers, particularly evident in the cerebral cortex.

The term “connectionists” comes from the idea that nodes in these models are interconnected. Instead of being explicitly programmed, these models learn their behavior through training. Deep learning is a connectionist approach.

Neural networks (NNs) consist of layers of interconnected nodes (neurons), each connection having an associated weight.

Neural networks process input data through these weighted connections, and learning occurs by adjusting the weights based on errors in the training data.

Attribution: LeNail, (2019). NN-SVG: Publication-Ready Neural Network Architecture Schematics. Journal of Open Source Software, 4(33), 747, https://doi.org/10.21105/joss.00747 (GitHub)

Hierarchy of concepts

In the book “Deep Learning” (Goodfellow, Bengio, and Courville 2016), authors Goodfellow, Bengio, and Courville define deep learning as a subset of machine learning that enables computers to “understand the world in terms of a hierarchy of concepts.”

This hierarchical approach is one of deep learning’s most significant contributions. It reduces the need for manual feature engineering and redirects the focus toward the engineering of neural network architectures.

Attribution: LeCun, Bengio, and Hinton (2015)

Basics

Computations with neurodes

where \(x_1, x_2 \in \{0,1\}\) and \(f(z)\) is an indicator function: \[ f(z)= \begin{cases}0, & z<\theta \\ 1, & z \geq \theta\end{cases} \]

In mathematics, \(f(z)\), as defined above, is known as an indicator function or a characteristic function.

These neurodes have one or more binary inputs, taking a value of 0 or 1, and one binary output.

They showed that such units could implement Boolean functions such as AND, OR, and NOT.

But also that networks of such units can compute any logical proposition.

McCulloch and Pitts (1943) termed artificial neurons, neurodes, for “neuron” + “node”.

Computations with neurodes

\[ y = f(x_1 + x_2)= \begin{cases}0, & x_1 + x_2 <\theta \\ 1, & x_1 + x_2 \geq \theta\end{cases} \]

• With \(\theta = 2\), the neurode implements an AND logic gate.

• With \(\theta = 1\), the neurode implements an OR logic gate.

With \(\theta = 1\), $x_1 {1} and \(x_2\) multiplied by (-1), \(y = 0\) when \(x_2 = 1\), \(y = 1\), if \(x_2 = 0\).

\[ y = f(x_1 + (-1) x_2)= \begin{cases}0, & x_1 + x_2 <\theta \\ 1, & x_1 + (-1) x_2 \geq \theta\end{cases} \]

Neurons can be broadly categorized into two primary types: excitatory and inhibitory.

More complex logic can be constructed by multiplying the inputs by -1, which is interpreted as inhibitory. Namely, this allows building a logical NOT.

Computations with neurodes

• Digital computations can be broken down into a sequence of logical operations, enabling neurode networks to execute any computation.

• McCulloch and Pitts (1943) did not focus on learning parameter \(\theta\).

• They introduced a machine that computes any function but cannot learn.

From this work, we take the idea that networks of such units perform computations. Signal propagates from one end of the network to compute a result.

Threshold logic unit

In 1957, Frank Rosenblatt developed a conceptually distinct model of a neuron known as the threshold logic unit, which he published in 1958.

In this model, both the inputs and the output of the neuron are represented as real values. Notably, each input connection has an associated weight.

The left section of the neuron, denoted by the sigma symbol, represents the computation of a weighted sum of its inputs, expressed as \(\theta_1 x_1 + \theta_2 x_2 + \ldots + \theta_D x_D + b\).

This sum is then processed through a step function, right section of the neuron, to generate the output.

Here, \(x^T \theta\) represents the dot product of two vectors: \(x\) and \(\theta\). Here, \(x^T\) denotes the transpose of the vector \(x\), converting it from a row vector to a column vector, allowing the dot product operation to be performed with the vector \(\theta\).

The dot product \(x^T \theta\) is then a scalar given by:

\[ x^T \theta = x^{(1)} \theta_1 + x^{(2)} \theta_2 + \cdots + x_{(D)} \theta_D \]

where \(x^{(j)}\) and \(theta_j\) are the components of the vectors \(x\) and \(\theta\), respectively.

Rosenblatt (1958)

Simple Step Functions

\(\text{heaviside}(t)\) =

• 1, if \(t \geq 0\)

• 0, if \(t < 0\)

\(\text{sign}(t)\) =

• 1, if \(t > 0\)

• 0, if \(t = 0\)

• -1, if \(t < 0\)

Common step functions include the heavyside function (0 if the input is negative and 1 otherwise) or the sign function (-1 if the input is negative, 0 if the input is zero, 1 otherwise).

Notation

Add an extra feature with a fixed value of 1 to the input. Associate it with weight \(b = \theta_{0}\), where \(b\) is the bias/intercept term.

Notation

The threshold logic unit is analogous to logistic regression, with the primary distinction being the substitution of the logistic (sigmoid) function with a step function. Similar to logistic regression, the perceptron is employed for classification tasks.

\(\theta_{0} = b\) is the bias/intercept term.

Perceptron

Since the threshold logic units in this single layer also generate the output, it is referred to as the output layer.

A perceptron consists of one or more threshold logic units arranged in a single layer, with each unit connected to all inputs. This configuration is referred to as fully connected or dense.

Perceptron

Classification tasks, can be further divided into multilabel and multiclass classification.

1. Multiclass Classification:

   • In multiclass classification, each instance is assigned to one and only one class out of a set of three or more possible classes. The classes are mutually exclusive, meaning that an instance cannot belong to more than one class at the same time.

   • Example: Classifying an image as either a cat, dog, or bird. Each image can only belong to one of these categories.

2. Multilabel Classification:

   • In multilabel classification, each instance can be associated with multiple classes simultaneously. The classes are not mutually exclusive, allowing for the possibility that an instance can belong to several classes at once.

   • Example: Tagging an image with multiple attributes such as “outdoor,” “sunset,” and “beach.” The image can simultaneously belong to all these labels.

The key difference lies in the relationship between classes: multiclass classification deals with a single label per instance, while multilabel classification handles multiple labels for each instance.

As this perceptron generates multiple outputs simultaneously, it performs multiple binary predictions, making it as a multilabel classifier (can also be used as multiclass classifier).

Notation

As before, introduce an additional feature with a value of 1 to the input. Assign a bias \(b\) to each neuron. Each incoming connection implicitly has an associated weight.

Notation

• \(X\) is the input data matrix where each row corresponds to an example and each column represents one of the \(D\) features.

• \(W\) is the weight matrix, structured with one row per input (feature) and one column per neuron.

• Bias terms can be represented separately; both approaches appear in the literature. Here, \(b\) is a vector with a length equal to the number of neurons.

With neural networks, the parameters of the model are often reffered to as \(w\) (vector) or \(W\) (matrix), rather than \(\theta\).

Discussion

• The algorithm to train the perceptron closely resembles stochastic gradient descent.

  • In the interest of time and to avoid confusion, we will skip this algorithm and focus on multilayer perception (MLP) and its training algorithm, backpropagation.

Historical Note and Justification

This limitation also applies to other linear classifiers, such as logistic regression.

Consequently, due to these limitations and a lack of practical applications, some researchers abandoned the perceptron.

Minsky and Papert (1969) demonstrated the limitations of perceptrons, notably their inability to solve exclusive OR (XOR) classification problems: \({([0,1],\mathrm{true}), ([1,0],\mathrm{true}), ([0,0],\mathrm{false}), ([1,1],\mathrm{false})}\).

Multilayer Perceptron

A multilayer perceptron (MLP) includes an input layer and one or more layers of threshold logic units. Layers that are neither input nor output are termed hidden layers.

XOR Classification problem

\(x^{(1)}\)	\(x^{(2)}\)	\(y\)	\(o_1\)	\(o_2\)	\(o_3\)
1	0	1	0	1	1
0	1	1	0	1	1
0	0	0	0	0	0
1	1	0	1	1	0

I developed an Excel spreadsheet to verify that the proposed multilayer perceptron effectively solves the XOR classification problem.

The step function used in the above model is the heavyside function.

\(x^{(1)}\) and \(x^{(2)}\) are two attributes, \(y\) is the target, \(o_1\), \(o_2\), and \(o_3 = h_\theta(x)\), are the output of the top left, bottom left, and right threshold units. Clearly \(h_\theta(x) = y, \forall x \in X\). The challenge during Rosenblatt’s time was the lack of algorithms to train multi-layer networks.

Feedforward Neural Network (FNN)

The network consists of three layers: input, hidden, and output. The input layer contains two nodes, the hidden layer comprises three nodes, and the output layer has two nodes. Additional hidden layers and nodes per layer can be added, which will be discussed later.

It is often useful to include explicit input nodes that do not perform calculations, known as input units or input neurons. These nodes act as placeholders to introduce input features into the network, passing data directly to the next layer without transformation. In the network diagram, these are the light blue nodes on the left, labeled 1 and 2. Typically, the number of input units corresponds to the number of features.

For clarity, nodes are labeled to facilitate discussion of the weights between them, such as \(w_{1,5}\) between nodes 1 and 5. Similarly, the output of a node is denoted by \(o_k\), where \(k\) represents the node’s label. For example, for \(k=3\), the output would be \(o_3\).

Information in this architecture flows unidirectionally—from left to right, moving from input to output. Consequently, it is termed a feedforward neural network.

Forward Pass (Computatation)

\(o3 = \sigma(w_{13} x^{(1)}+ w_{23} x^{(2)} + b_3)\)

\(o4 = \sigma(w_{14} x^{(1)}+ w_{24} x^{(2)} + b_4)\)

\(o5 = \sigma(w_{15} x^{(1)}+ w_{25} x^{(2)} + b_5)\)

\(o6 = \sigma(w_{36} o_3 + w_{46} o_4 + w_{56} o_5 + b_6)\)

\(o7 = \sigma(w_{37} o_3 + w_{47} o_4 + w_{57} o_5 + b_7)\)

To simplify the figure, I have opted not to display the bias terms, though they remain crucial components. Specifically, \(b_3\) represents the bias term associated with node 3.

If bias terms were not significant, the training process would naturally reduce them to zero. Bias terms are essential as they enable the adjustment of the decision boundary, allowing the model to learn more complex patterns that weights alone cannot capture. By offering additional degrees of freedom, they also contribute to faster convergence during training.

First, it’s important to understand the information flow: this network computes two outputs from its inputs.

Forward Pass (Computatation)

import numpy as np

# Sigmoid function

def sigma(x):
    return 1 / (1 + np.exp(-x))

# Input (two attributes) vector, one example of our trainig set

x1, x2 = (0.5, 0.9)

# Initializing the weights of layers 2 and 3 to random values

w13, w14, w15, w23, w24, w25 = np.random.uniform(low=-1, high=1, size=6)
w36, w46, w56, w37, w47, w57 = np.random.uniform(low=-1, high=1, size=6)

# Initializing all 5 bias terms to random values

b3, b4, b5, b6, b7 = np.random.uniform(low=-1, high=1, size=5)

o3 = sigma(w13 * x1 + w23 * x2 + b3)
o4 = sigma(w14 * x1 + w24 * x2 + b4)
o5 = sigma(w15 * x1 + w25 * x2 + b5)
o6 = sigma(w36 * o3 + w46 * o4 + w56 * o5 + b6)
o7 = sigma(w37 * o3 + w47 * o4 + w57 * o5 + b7)

(o6, o7)

(0.38440224844051585, 0.16368370458391393)

The example above illustrates the computation process with specific values. Before training a neural network, it is standard practice to initialize the weights and biases with random values. Gradient descent is then employed to iteratively adjust these parameters, aiming to minimize the loss function.

Forward Pass (Computatation)

The information flow remains consistent even in more complex networks. Networks with many layers are called deep neural networks (DNN).

Produced using NN-SVG, LeNail (2019).

Forward Pass (Computatation)

Same network with bias terms shown.

Produced using NN-SVG, LeNail (2019).

Activation Function

• As will be discussed later, the training algorithm, known as backpropagation, employs gradient descent, necessitating the calculation of the partial derivatives of the loss function.

• The step function in the multilayer perceptron had to be replaced, as it consists only of flat surfaces. Gradient descent cannot progress on flat surfaces due to their zero derivative.

Activation Function

• Nonlinear activation functions are paramount because, without them, multiple layers in the network would only compute a linear function of the inputs.

• According to the Universal Approximation Theorem, sufficiently large deep networks with nonlinear activation functions can approximate any continuous function. See Universal Approximation Theorem.

Sigmoid

\[ \sigma(t) = \frac{1}{1 + e^{-t}} \]

Hyperbolic Tangent Function

\[ \tanh(t) = 2 \sigma(2t) - 1 \]

This S-shaped curve, similar to the sigmoid function, produces output values ranging from -1 to 1. According to Géron (2022), this range helps each layer’s output to be approximately centered around 0 at the start of training, thereby accelerating convergence.

Rectified linear unit function (ReLU)

\[ \mathrm{ReLU}(t) = \max(0, t) \]

Although the ReLU function is not differentiable at \(t=0\) and has a derivative of 0 for \(t<0\), it performs quite well in practice and is computationally efficient. Consequently, it has become the default activation function.

Common Activation Functions

Géron (2022) – 10_neural_nets_with_keras.ipynb

Universal Approximation

Definition

The universal approximation theorem (UAT) states that a feedforward neural network with a single hidden layer containing a finite number of neurons can approximate any continuous function on a compact subset of \(\mathbb{R}^n\), given appropriate weights and activation functions.

In mathematical terms, a subset of \(\mathbb{R}^n\) is considered compact if it is both closed and bounded.

• Closed: A set is closed if it contains all its boundary points. In other words, it includes its limit points or accumulation points.

• Bounded: A set is bounded if there exists a real number (M) such that the distance between any two points in the set is less than \(M\).

In the context of the universal approximation theorem, compactness ensures that the function being approximated is defined on a finite and well-behaved region, which is crucial for the theoretical guarantees provided by the theorem.

Cybenko (1989); Hornik, Stinchcombe, and White (1989)

Demonstration with code

import numpy as np

# Defining the function to be approximated

def f(x):
  return 2 * x**3 + 4 * x**2 - 5 * x + 1

# Generating a dataset, x in [-4,2), f(x) as above

X = 6 * np.random.rand(1000, 1) - 4

y = f(X.flatten())

Increasing the number of neurons

from sklearn.neural_network import MLPRegressor
from sklearn.model_selection import train_test_split

X_train, X_valid, y_train, y_valid = train_test_split(X, y, test_size=0.1, random_state=42)

models = []

sizes = [1, 2, 5, 10, 100]

for i, n in enumerate(sizes):

  models.append(MLPRegressor(hidden_layer_sizes=[n], max_iter=5000, random_state=42))

  models[i].fit(X_train, y_train) 

MLPRegressor is a multi-layer perceptron regressor from sklearn. Its default activation function is relu.

Increasing the number of neurons

In the example above, I retained only 10% of the data as the test set because the function being approximated is straightforward and noise-free. This decision was made to ensure that the true curve does not overshadow the other results.

Increasing the number of neurons

As expected, increasing neuron count reduces loss.

Universal Approximation

The video effectively elucidates key concepts (terminology) in neural networks, including nodes, layers, weights, and activation functions. It demonstrates the process of summing activation outputs from a preceding layer, akin to the aggregation of curves. Additionally, the video illustrates how scaling an output by a weight not only alters the amplitude of a curve but also inverts its orientation when the weight is negative. Moreover, it clearly depicts the function of bias terms in vertically shifting the curve, contingent on the sign of the bias.

This video effectively conveys the underlying intuition of the universal approximation theorem. (18m 53s)

Let’s code

Frameworks

PyTorch and TensorFlow are the leading platforms for deep learning.

• PyTorch has gained considerable traction in the research community. Initially developed by Meta AI, it is now part of the Linux Foundation.

• TensorFlow, created by Google, is widely adopted in industry for deploying models in production environments.

Keras

Keras is a high-level API designed to build, train, evaluate, and execute models across various backends, including PyTorch, TensorFlow, and JAX, Google’s high-performance platform.

As highlighted in previous Quotes of the Day, François Chollet, a Google engineer, is the originator and one of the primary developers of the Keras project.

Keras is powerful enough for most projects.

Fashion-MNIST dataset

“Fashion-MNIST is a dataset of Zalando’s article images—consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grayscale image, associated with a label from 10 classes.”

Attribution: Géron (2022) – 10_neural_nets_with_keras.ipynb

Loading

import tensorflow as tf

fashion_mnist = tf.keras.datasets.fashion_mnist.load_data()

(X_train_full, y_train_full), (X_test, y_test) = fashion_mnist

X_train, y_train = X_train_full[:-5000], y_train_full[:-5000]
X_valid, y_valid = X_train_full[-5000:], y_train_full[-5000:]

Setting aside 5000 examples as a validation set.

Exploration

X_train.shape

(55000, 28, 28)

X_train.dtype

dtype('uint8')

Transforming the pixel intensities from integers in the range 0 to 255 to floats in the range 0 to 1.

X_train, X_valid, X_test = X_train / 255., X_valid / 255., X_test / 255.

What are these images anyway!

plt.figure(figsize=(2, 2))
plt.imshow(X_train[0], cmap="binary")
plt.axis('off')
plt.show()

y_train

array([9, 0, 0, ..., 9, 0, 2], dtype=uint8)

Since the labels are integers, 0 to 9. Class names will become handy.

class_names = ["T-shirt/top", "Trouser", "Pullover", "Dress", "Coat",
               "Sandal", "Shirt", "Sneaker", "Bag", "Ankle boot"]

First 40 images

n_rows = 4
n_cols = 10
plt.figure(figsize=(n_cols * 1.2, n_rows * 1.2))
for row in range(n_rows):
    for col in range(n_cols):
        index = n_cols * row + col
        plt.subplot(n_rows, n_cols, index + 1)
        plt.imshow(X_train[index], cmap="binary", interpolation="nearest")
        plt.axis('off')
        plt.title(class_names[y_train[index]])
plt.subplots_adjust(wspace=0.2, hspace=0.5)
plt.show()

First 40 images

Creating a model

tf.random.set_seed(42)

model = tf.keras.Sequential()

model.add(tf.keras.layers.InputLayer(shape=[28, 28]))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(300, activation="relu"))
model.add(tf.keras.layers.Dense(100, activation="relu"))
model.add(tf.keras.layers.Dense(10, activation="softmax"))

model.summary()

Model: "sequential"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ flatten (Flatten)               │ (None, 784)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense (Dense)                   │ (None, 300)            │       235,500 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense)                 │ (None, 100)            │        30,100 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_2 (Dense)                 │ (None, 10)             │         1,010 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 266,610 (1.02 MB)

 Trainable params: 266,610 (1.02 MB)

 Non-trainable params: 0 (0.00 B)

As observed, dense_3 has \(235,500\) parameters, while \(784 \times 300 = 235,200\).

Could you explain the origin of the additional parameters?

Similarly, dense_3 has \(30,100\) parameters, while \(300 \times 100 = 30,000\).

Can you explain why?

Creating a model (alternative)

model = tf.keras.Sequential([
    tf.keras.layers.Flatten(input_shape=[28, 28]),
    tf.keras.layers.Dense(300, activation="relu"),
    tf.keras.layers.Dense(100, activation="relu"),
    tf.keras.layers.Dense(10, activation="softmax")
])

model.summary()

Model: "sequential"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Layer (type)                    ┃ Output Shape           ┃       Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ flatten (Flatten)               │ (None, 784)            │             0 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense (Dense)                   │ (None, 300)            │       235,500 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_1 (Dense)                 │ (None, 100)            │        30,100 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_2 (Dense)                 │ (None, 10)             │         1,010 │
└─────────────────────────────────┴────────────────────────┴───────────────┘

 Total params: 266,610 (1.02 MB)

 Trainable params: 266,610 (1.02 MB)

 Non-trainable params: 0 (0.00 B)

Compiling the model

model.compile(loss="sparse_categorical_crossentropy",
              optimizer="sgd",
              metrics=["accuracy"])

sparse_categorical_crossentropy is the appropriate function for a multiclass classification problem (more later).

The method compile allows to set the loss function, as well as other parameters. Keras then prepares the model for training.

Training the model

history = model.fit(X_train, y_train, epochs=30,
                    validation_data=(X_valid, y_valid))

Epoch 1/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 3:37 127ms/step - accuracy: 0.0938 - loss: 2.3124  58/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 882us/step - accuracy: 0.3576 - loss: 2.0430   122/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 835us/step - accuracy: 0.4337 - loss: 1.8658 188/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 809us/step - accuracy: 0.4795 - loss: 1.7324 253/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 799us/step - accuracy: 0.5119 - loss: 1.6306 317/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 796us/step - accuracy: 0.5352 - loss: 1.5507 382/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 792us/step - accuracy: 0.5541 - loss: 1.4838 447/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 790us/step - accuracy: 0.5697 - loss: 1.4277 511/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 789us/step - accuracy: 0.5827 - loss: 1.3803 576/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 787us/step - accuracy: 0.5942 - loss: 1.3385 641/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 786us/step - accuracy: 0.6043 - loss: 1.3016 707/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 784us/step - accuracy: 0.6133 - loss: 1.2683 772/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 783us/step - accuracy: 0.6213 - loss: 1.2389 836/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 784us/step - accuracy: 0.6285 - loss: 1.2126 901/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 783us/step - accuracy: 0.6352 - loss: 1.1882 967/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 782us/step - accuracy: 0.6414 - loss: 1.16531034/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 780us/step - accuracy: 0.6473 - loss: 1.14391100/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 779us/step - accuracy: 0.6526 - loss: 1.12451165/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 779us/step - accuracy: 0.6575 - loss: 1.10671230/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 779us/step - accuracy: 0.6620 - loss: 1.09021297/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 778us/step - accuracy: 0.6663 - loss: 1.07431363/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 777us/step - accuracy: 0.6703 - loss: 1.05961427/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 777us/step - accuracy: 0.6741 - loss: 1.04611492/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 777us/step - accuracy: 0.6776 - loss: 1.03321556/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 778us/step - accuracy: 0.6809 - loss: 1.02111617/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 780us/step - accuracy: 0.6839 - loss: 1.01021680/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 780us/step - accuracy: 0.6869 - loss: 0.99951719/1719 ━━━━━━━━━━━━━━━━━━━━ 2s 876us/step - accuracy: 0.6886 - loss: 0.9930 - val_accuracy: 0.8294 - val_loss: 0.5042
Epoch 2/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 18s 11ms/step - accuracy: 0.8750 - loss: 0.5813  65/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 788us/step - accuracy: 0.8366 - loss: 0.5071 129/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 786us/step - accuracy: 0.8283 - loss: 0.5181 193/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 785us/step - accuracy: 0.8244 - loss: 0.5232 258/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 783us/step - accuracy: 0.8229 - loss: 0.5237 321/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 788us/step - accuracy: 0.8217 - loss: 0.5239 385/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 788us/step - accuracy: 0.8211 - loss: 0.5236 450/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 786us/step - accuracy: 0.8207 - loss: 0.5232 515/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 785us/step - accuracy: 0.8204 - loss: 0.5226 578/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 787us/step - accuracy: 0.8204 - loss: 0.5220 641/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 788us/step - accuracy: 0.8205 - loss: 0.5213 703/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 790us/step - accuracy: 0.8207 - loss: 0.5206 767/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 790us/step - accuracy: 0.8209 - loss: 0.5199 830/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 790us/step - accuracy: 0.8211 - loss: 0.5192 893/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8214 - loss: 0.5186 956/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8217 - loss: 0.51771020/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8220 - loss: 0.51681083/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.8223 - loss: 0.51591146/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.8226 - loss: 0.51501209/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.8229 - loss: 0.51421274/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.8231 - loss: 0.51351338/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8234 - loss: 0.51271401/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.8237 - loss: 0.51191464/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.8240 - loss: 0.51111527/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.8243 - loss: 0.51041590/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 793us/step - accuracy: 0.8245 - loss: 0.50961653/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 793us/step - accuracy: 0.8248 - loss: 0.50891718/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 793us/step - accuracy: 0.8250 - loss: 0.50821719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 850us/step - accuracy: 0.8250 - loss: 0.5082 - val_accuracy: 0.8408 - val_loss: 0.4527
Epoch 3/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 18s 11ms/step - accuracy: 0.8750 - loss: 0.5051  64/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 800us/step - accuracy: 0.8534 - loss: 0.4414 126/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 804us/step - accuracy: 0.8467 - loss: 0.4534 190/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 797us/step - accuracy: 0.8436 - loss: 0.4603 255/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 792us/step - accuracy: 0.8426 - loss: 0.4615 319/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 791us/step - accuracy: 0.8418 - loss: 0.4624 384/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 789us/step - accuracy: 0.8413 - loss: 0.4629 447/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 791us/step - accuracy: 0.8408 - loss: 0.4631 510/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8404 - loss: 0.4631 573/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.8403 - loss: 0.4631 637/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.8404 - loss: 0.4629 700/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.8404 - loss: 0.4626 764/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.8405 - loss: 0.4624 827/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.8406 - loss: 0.4622 890/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.8407 - loss: 0.4620 954/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.8408 - loss: 0.46161018/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8410 - loss: 0.46111079/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 793us/step - accuracy: 0.8411 - loss: 0.46061143/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 793us/step - accuracy: 0.8413 - loss: 0.46011206/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 794us/step - accuracy: 0.8414 - loss: 0.45971270/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 793us/step - accuracy: 0.8415 - loss: 0.45931334/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 793us/step - accuracy: 0.8416 - loss: 0.45891399/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.8418 - loss: 0.45841465/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8419 - loss: 0.45801529/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8420 - loss: 0.45751592/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8422 - loss: 0.45711655/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8423 - loss: 0.45671718/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8424 - loss: 0.45631719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 848us/step - accuracy: 0.8424 - loss: 0.4563 - val_accuracy: 0.8470 - val_loss: 0.4283
Epoch 4/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 18s 11ms/step - accuracy: 0.8438 - loss: 0.4670  64/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 794us/step - accuracy: 0.8622 - loss: 0.4074 125/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 807us/step - accuracy: 0.8573 - loss: 0.4192 187/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 808us/step - accuracy: 0.8539 - loss: 0.4267 253/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 798us/step - accuracy: 0.8524 - loss: 0.4284 318/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 794us/step - accuracy: 0.8513 - loss: 0.4296 382/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 792us/step - accuracy: 0.8507 - loss: 0.4302 447/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 790us/step - accuracy: 0.8501 - loss: 0.4308 510/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8497 - loss: 0.4311 574/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8495 - loss: 0.4312 639/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 789us/step - accuracy: 0.8495 - loss: 0.4311 703/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 789us/step - accuracy: 0.8496 - loss: 0.4310 767/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 789us/step - accuracy: 0.8497 - loss: 0.4309 832/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 788us/step - accuracy: 0.8497 - loss: 0.4308 896/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 787us/step - accuracy: 0.8498 - loss: 0.4308 961/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 787us/step - accuracy: 0.8499 - loss: 0.43051025/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 787us/step - accuracy: 0.8500 - loss: 0.43021088/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 788us/step - accuracy: 0.8502 - loss: 0.42981154/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 786us/step - accuracy: 0.8503 - loss: 0.42951219/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 786us/step - accuracy: 0.8504 - loss: 0.42911285/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 784us/step - accuracy: 0.8505 - loss: 0.42891350/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 784us/step - accuracy: 0.8506 - loss: 0.42851416/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 783us/step - accuracy: 0.8508 - loss: 0.42821484/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 781us/step - accuracy: 0.8509 - loss: 0.42781549/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 781us/step - accuracy: 0.8511 - loss: 0.42751614/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 780us/step - accuracy: 0.8512 - loss: 0.42721680/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 780us/step - accuracy: 0.8513 - loss: 0.42691719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 836us/step - accuracy: 0.8513 - loss: 0.4267 - val_accuracy: 0.8520 - val_loss: 0.4135
Epoch 5/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 18s 11ms/step - accuracy: 0.8438 - loss: 0.4403  65/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 788us/step - accuracy: 0.8655 - loss: 0.3843 129/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 787us/step - accuracy: 0.8609 - loss: 0.3968 195/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 778us/step - accuracy: 0.8583 - loss: 0.4041 260/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 777us/step - accuracy: 0.8575 - loss: 0.4058 326/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 775us/step - accuracy: 0.8569 - loss: 0.4070 391/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 776us/step - accuracy: 0.8566 - loss: 0.4079 457/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 773us/step - accuracy: 0.8563 - loss: 0.4085 523/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 772us/step - accuracy: 0.8561 - loss: 0.4089 589/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 772us/step - accuracy: 0.8560 - loss: 0.4091 655/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 770us/step - accuracy: 0.8561 - loss: 0.4090 721/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 770us/step - accuracy: 0.8562 - loss: 0.4089 787/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 770us/step - accuracy: 0.8562 - loss: 0.4089 852/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 770us/step - accuracy: 0.8563 - loss: 0.4088 918/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 770us/step - accuracy: 0.8564 - loss: 0.4088 982/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 771us/step - accuracy: 0.8565 - loss: 0.40861048/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 770us/step - accuracy: 0.8566 - loss: 0.40831113/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 770us/step - accuracy: 0.8567 - loss: 0.40791178/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 770us/step - accuracy: 0.8569 - loss: 0.40761244/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 770us/step - accuracy: 0.8570 - loss: 0.40741312/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 769us/step - accuracy: 0.8571 - loss: 0.40711377/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 769us/step - accuracy: 0.8572 - loss: 0.40681442/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 769us/step - accuracy: 0.8573 - loss: 0.40651508/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 769us/step - accuracy: 0.8575 - loss: 0.40621574/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 769us/step - accuracy: 0.8576 - loss: 0.40591639/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 769us/step - accuracy: 0.8577 - loss: 0.40571706/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 769us/step - accuracy: 0.8578 - loss: 0.40541719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 825us/step - accuracy: 0.8578 - loss: 0.4054 - val_accuracy: 0.8532 - val_loss: 0.4012
Epoch 6/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 18s 11ms/step - accuracy: 0.8750 - loss: 0.4133  66/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 776us/step - accuracy: 0.8718 - loss: 0.3645 133/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 766us/step - accuracy: 0.8676 - loss: 0.3779 199/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 764us/step - accuracy: 0.8651 - loss: 0.3852 266/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 760us/step - accuracy: 0.8643 - loss: 0.3872 332/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 762us/step - accuracy: 0.8638 - loss: 0.3884 398/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 762us/step - accuracy: 0.8635 - loss: 0.3895 463/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 763us/step - accuracy: 0.8633 - loss: 0.3901 529/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 764us/step - accuracy: 0.8631 - loss: 0.3906 594/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 765us/step - accuracy: 0.8630 - loss: 0.3908 659/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 766us/step - accuracy: 0.8630 - loss: 0.3908 725/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 766us/step - accuracy: 0.8630 - loss: 0.3908 790/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 766us/step - accuracy: 0.8630 - loss: 0.3908 856/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 766us/step - accuracy: 0.8630 - loss: 0.3908 922/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 766us/step - accuracy: 0.8630 - loss: 0.3908 988/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 765us/step - accuracy: 0.8631 - loss: 0.39061053/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 766us/step - accuracy: 0.8632 - loss: 0.39041120/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 765us/step - accuracy: 0.8633 - loss: 0.39011187/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 764us/step - accuracy: 0.8633 - loss: 0.38981253/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 764us/step - accuracy: 0.8634 - loss: 0.38961317/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 765us/step - accuracy: 0.8635 - loss: 0.38941384/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 764us/step - accuracy: 0.8635 - loss: 0.38921450/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 764us/step - accuracy: 0.8636 - loss: 0.38891516/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 764us/step - accuracy: 0.8637 - loss: 0.38861580/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 765us/step - accuracy: 0.8638 - loss: 0.38841645/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 765us/step - accuracy: 0.8639 - loss: 0.38821711/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 765us/step - accuracy: 0.8639 - loss: 0.38801719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 822us/step - accuracy: 0.8639 - loss: 0.3879 - val_accuracy: 0.8590 - val_loss: 0.3925
Epoch 7/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 18s 11ms/step - accuracy: 0.8438 - loss: 0.3960  64/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 800us/step - accuracy: 0.8691 - loss: 0.3489 130/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 784us/step - accuracy: 0.8673 - loss: 0.3618 192/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 792us/step - accuracy: 0.8657 - loss: 0.3694 255/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 795us/step - accuracy: 0.8656 - loss: 0.3717 319/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 794us/step - accuracy: 0.8655 - loss: 0.3732 383/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 793us/step - accuracy: 0.8657 - loss: 0.3742 447/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 793us/step - accuracy: 0.8656 - loss: 0.3751 511/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 793us/step - accuracy: 0.8655 - loss: 0.3756 575/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8655 - loss: 0.3760 639/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8657 - loss: 0.3761 703/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 790us/step - accuracy: 0.8658 - loss: 0.3760 767/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 790us/step - accuracy: 0.8659 - loss: 0.3760 831/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 790us/step - accuracy: 0.8660 - loss: 0.3760 894/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8661 - loss: 0.3762 958/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 790us/step - accuracy: 0.8662 - loss: 0.37611021/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8664 - loss: 0.37591085/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8665 - loss: 0.37561149/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8667 - loss: 0.37541213/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8668 - loss: 0.37521277/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 790us/step - accuracy: 0.8669 - loss: 0.37501340/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8670 - loss: 0.37481403/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8671 - loss: 0.37461466/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8673 - loss: 0.37441530/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8674 - loss: 0.37411593/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.8675 - loss: 0.37391655/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.8676 - loss: 0.37371719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 846us/step - accuracy: 0.8677 - loss: 0.3735 - val_accuracy: 0.8606 - val_loss: 0.3850
Epoch 8/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 18s 11ms/step - accuracy: 0.9062 - loss: 0.3761  63/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 811us/step - accuracy: 0.8788 - loss: 0.3350 128/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 791us/step - accuracy: 0.8744 - loss: 0.3478 193/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 785us/step - accuracy: 0.8718 - loss: 0.3560 258/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 784us/step - accuracy: 0.8713 - loss: 0.3585 323/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 781us/step - accuracy: 0.8709 - loss: 0.3600 388/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 779us/step - accuracy: 0.8708 - loss: 0.3611 454/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 777us/step - accuracy: 0.8706 - loss: 0.3620 520/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 775us/step - accuracy: 0.8705 - loss: 0.3626 584/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 776us/step - accuracy: 0.8704 - loss: 0.3630 650/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 775us/step - accuracy: 0.8705 - loss: 0.3630 715/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 775us/step - accuracy: 0.8706 - loss: 0.3630 780/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 775us/step - accuracy: 0.8707 - loss: 0.3631 844/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 776us/step - accuracy: 0.8708 - loss: 0.3631 909/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 776us/step - accuracy: 0.8708 - loss: 0.3632 975/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 776us/step - accuracy: 0.8710 - loss: 0.36311041/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 775us/step - accuracy: 0.8711 - loss: 0.36301105/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 775us/step - accuracy: 0.8712 - loss: 0.36271170/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 775us/step - accuracy: 0.8714 - loss: 0.36251234/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 776us/step - accuracy: 0.8715 - loss: 0.36241299/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 776us/step - accuracy: 0.8716 - loss: 0.36221362/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 777us/step - accuracy: 0.8717 - loss: 0.36201427/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 777us/step - accuracy: 0.8718 - loss: 0.36181493/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 776us/step - accuracy: 0.8720 - loss: 0.36161557/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 776us/step - accuracy: 0.8721 - loss: 0.36141621/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 777us/step - accuracy: 0.8722 - loss: 0.36121687/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 776us/step - accuracy: 0.8722 - loss: 0.36101719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 833us/step - accuracy: 0.8723 - loss: 0.3609 - val_accuracy: 0.8608 - val_loss: 0.3781
Epoch 9/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 18s 11ms/step - accuracy: 0.9062 - loss: 0.3640  64/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 803us/step - accuracy: 0.8843 - loss: 0.3230 131/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 776us/step - accuracy: 0.8794 - loss: 0.3359 196/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 777us/step - accuracy: 0.8768 - loss: 0.3438 260/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 780us/step - accuracy: 0.8761 - loss: 0.3463 324/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 781us/step - accuracy: 0.8757 - loss: 0.3478 389/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 779us/step - accuracy: 0.8754 - loss: 0.3490 455/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 777us/step - accuracy: 0.8752 - loss: 0.3499 520/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 777us/step - accuracy: 0.8750 - loss: 0.3506 584/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 778us/step - accuracy: 0.8748 - loss: 0.3510 650/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 776us/step - accuracy: 0.8748 - loss: 0.3510 714/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 777us/step - accuracy: 0.8748 - loss: 0.3511 779/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 777us/step - accuracy: 0.8749 - loss: 0.3512 844/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 777us/step - accuracy: 0.8749 - loss: 0.3513 908/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 778us/step - accuracy: 0.8749 - loss: 0.3514 975/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 776us/step - accuracy: 0.8750 - loss: 0.35141040/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 776us/step - accuracy: 0.8751 - loss: 0.35121105/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 776us/step - accuracy: 0.8752 - loss: 0.35101171/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 775us/step - accuracy: 0.8753 - loss: 0.35081236/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 775us/step - accuracy: 0.8754 - loss: 0.35071302/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 774us/step - accuracy: 0.8755 - loss: 0.35061368/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 774us/step - accuracy: 0.8756 - loss: 0.35041434/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 773us/step - accuracy: 0.8756 - loss: 0.35021499/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 773us/step - accuracy: 0.8757 - loss: 0.35001564/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 774us/step - accuracy: 0.8758 - loss: 0.34981630/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 773us/step - accuracy: 0.8759 - loss: 0.34971694/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 774us/step - accuracy: 0.8759 - loss: 0.34951719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 830us/step - accuracy: 0.8759 - loss: 0.3495 - val_accuracy: 0.8638 - val_loss: 0.3736
Epoch 10/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 18s 11ms/step - accuracy: 0.9062 - loss: 0.3505  65/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 786us/step - accuracy: 0.8835 - loss: 0.3136 131/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 772us/step - accuracy: 0.8798 - loss: 0.3258 195/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 777us/step - accuracy: 0.8776 - loss: 0.3335 261/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 773us/step - accuracy: 0.8773 - loss: 0.3360 326/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 774us/step - accuracy: 0.8770 - loss: 0.3375 391/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 775us/step - accuracy: 0.8769 - loss: 0.3387 457/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 773us/step - accuracy: 0.8769 - loss: 0.3397 522/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 773us/step - accuracy: 0.8768 - loss: 0.3403 588/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 772us/step - accuracy: 0.8768 - loss: 0.3407 654/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 771us/step - accuracy: 0.8769 - loss: 0.3408 719/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 772us/step - accuracy: 0.8770 - loss: 0.3408 786/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 770us/step - accuracy: 0.8771 - loss: 0.3409 850/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 771us/step - accuracy: 0.8772 - loss: 0.3411 915/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 771us/step - accuracy: 0.8772 - loss: 0.3412 981/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 771us/step - accuracy: 0.8773 - loss: 0.34121049/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 769us/step - accuracy: 0.8775 - loss: 0.34101116/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 768us/step - accuracy: 0.8776 - loss: 0.34091183/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 768us/step - accuracy: 0.8778 - loss: 0.34071249/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 768us/step - accuracy: 0.8779 - loss: 0.34061315/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 767us/step - accuracy: 0.8780 - loss: 0.34041383/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 766us/step - accuracy: 0.8782 - loss: 0.34031449/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 766us/step - accuracy: 0.8783 - loss: 0.34011515/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 766us/step - accuracy: 0.8784 - loss: 0.33991580/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 766us/step - accuracy: 0.8785 - loss: 0.33981644/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 767us/step - accuracy: 0.8786 - loss: 0.33961709/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 767us/step - accuracy: 0.8786 - loss: 0.33951719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 823us/step - accuracy: 0.8786 - loss: 0.3395 - val_accuracy: 0.8644 - val_loss: 0.3687
Epoch 11/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 18s 11ms/step - accuracy: 0.9062 - loss: 0.3378  65/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 789us/step - accuracy: 0.8856 - loss: 0.3035 130/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 782us/step - accuracy: 0.8827 - loss: 0.3154 196/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 776us/step - accuracy: 0.8804 - loss: 0.3234 261/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 775us/step - accuracy: 0.8800 - loss: 0.3261 327/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 773us/step - accuracy: 0.8797 - loss: 0.3277 392/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 773us/step - accuracy: 0.8797 - loss: 0.3290 459/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 770us/step - accuracy: 0.8796 - loss: 0.3300 522/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 773us/step - accuracy: 0.8796 - loss: 0.3307 586/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 774us/step - accuracy: 0.8796 - loss: 0.3311 651/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 774us/step - accuracy: 0.8797 - loss: 0.3312 716/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 774us/step - accuracy: 0.8798 - loss: 0.3313 783/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 773us/step - accuracy: 0.8799 - loss: 0.3314 848/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 773us/step - accuracy: 0.8800 - loss: 0.3316 914/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 772us/step - accuracy: 0.8800 - loss: 0.3318 979/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 772us/step - accuracy: 0.8802 - loss: 0.33171044/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 772us/step - accuracy: 0.8803 - loss: 0.33161110/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 772us/step - accuracy: 0.8804 - loss: 0.33151176/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 771us/step - accuracy: 0.8806 - loss: 0.33131242/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 770us/step - accuracy: 0.8807 - loss: 0.33121307/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 771us/step - accuracy: 0.8808 - loss: 0.33111372/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 771us/step - accuracy: 0.8809 - loss: 0.33101437/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 771us/step - accuracy: 0.8810 - loss: 0.33081503/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 770us/step - accuracy: 0.8811 - loss: 0.33071569/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 770us/step - accuracy: 0.8812 - loss: 0.33051632/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 771us/step - accuracy: 0.8813 - loss: 0.33041696/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 772us/step - accuracy: 0.8814 - loss: 0.33031719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 829us/step - accuracy: 0.8814 - loss: 0.3302 - val_accuracy: 0.8648 - val_loss: 0.3654
Epoch 12/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 18s 11ms/step - accuracy: 0.9062 - loss: 0.3286  67/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 767us/step - accuracy: 0.8936 - loss: 0.2948 130/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 783us/step - accuracy: 0.8893 - loss: 0.3062 196/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 778us/step - accuracy: 0.8864 - loss: 0.3143 263/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 771us/step - accuracy: 0.8856 - loss: 0.3170 328/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 771us/step - accuracy: 0.8850 - loss: 0.3185 393/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 772us/step - accuracy: 0.8847 - loss: 0.3199 457/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 773us/step - accuracy: 0.8845 - loss: 0.3209 523/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 772us/step - accuracy: 0.8842 - loss: 0.3217 588/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 773us/step - accuracy: 0.8840 - loss: 0.3222 652/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 774us/step - accuracy: 0.8840 - loss: 0.3223 718/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 773us/step - accuracy: 0.8840 - loss: 0.3224 781/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 775us/step - accuracy: 0.8840 - loss: 0.3225 846/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 775us/step - accuracy: 0.8840 - loss: 0.3227 911/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 775us/step - accuracy: 0.8840 - loss: 0.3229 976/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 775us/step - accuracy: 0.8840 - loss: 0.32291043/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 774us/step - accuracy: 0.8841 - loss: 0.32281108/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 773us/step - accuracy: 0.8842 - loss: 0.32271173/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 774us/step - accuracy: 0.8843 - loss: 0.32261240/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 773us/step - accuracy: 0.8844 - loss: 0.32251305/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 773us/step - accuracy: 0.8845 - loss: 0.32241371/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 772us/step - accuracy: 0.8846 - loss: 0.32231437/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 772us/step - accuracy: 0.8847 - loss: 0.32211504/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 771us/step - accuracy: 0.8848 - loss: 0.32201570/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 771us/step - accuracy: 0.8848 - loss: 0.32191635/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 771us/step - accuracy: 0.8849 - loss: 0.32181702/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 770us/step - accuracy: 0.8849 - loss: 0.32171719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 826us/step - accuracy: 0.8850 - loss: 0.3216 - val_accuracy: 0.8676 - val_loss: 0.3609
Epoch 13/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 18s 11ms/step - accuracy: 0.9062 - loss: 0.3127  62/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 824us/step - accuracy: 0.8974 - loss: 0.2862 120/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 845us/step - accuracy: 0.8937 - loss: 0.2959 178/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 855us/step - accuracy: 0.8904 - loss: 0.3048 235/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 863us/step - accuracy: 0.8891 - loss: 0.3079 294/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 861us/step - accuracy: 0.8885 - loss: 0.3097 350/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 867us/step - accuracy: 0.8883 - loss: 0.3108 410/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 864us/step - accuracy: 0.8880 - loss: 0.3121 469/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 862us/step - accuracy: 0.8878 - loss: 0.3130 532/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 855us/step - accuracy: 0.8875 - loss: 0.3137 593/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 851us/step - accuracy: 0.8874 - loss: 0.3142 656/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 846us/step - accuracy: 0.8874 - loss: 0.3143 718/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 844us/step - accuracy: 0.8873 - loss: 0.3144 781/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 840us/step - accuracy: 0.8873 - loss: 0.3146 845/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 835us/step - accuracy: 0.8873 - loss: 0.3148 909/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 832us/step - accuracy: 0.8873 - loss: 0.3150 973/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 829us/step - accuracy: 0.8874 - loss: 0.31501037/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 826us/step - accuracy: 0.8875 - loss: 0.31501100/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 825us/step - accuracy: 0.8875 - loss: 0.31491166/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 821us/step - accuracy: 0.8877 - loss: 0.31471229/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 820us/step - accuracy: 0.8877 - loss: 0.31471293/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 819us/step - accuracy: 0.8878 - loss: 0.31461358/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 817us/step - accuracy: 0.8879 - loss: 0.31451421/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 816us/step - accuracy: 0.8880 - loss: 0.31431485/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 815us/step - accuracy: 0.8881 - loss: 0.31421550/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 813us/step - accuracy: 0.8882 - loss: 0.31411615/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 812us/step - accuracy: 0.8882 - loss: 0.31401679/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 810us/step - accuracy: 0.8883 - loss: 0.31391719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 866us/step - accuracy: 0.8883 - loss: 0.3139 - val_accuracy: 0.8690 - val_loss: 0.3571
Epoch 14/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 18s 11ms/step - accuracy: 0.9062 - loss: 0.2941  65/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 792us/step - accuracy: 0.9007 - loss: 0.2779 129/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 792us/step - accuracy: 0.8968 - loss: 0.2891 195/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 781us/step - accuracy: 0.8938 - loss: 0.2974 260/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 779us/step - accuracy: 0.8927 - loss: 0.3003 325/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 777us/step - accuracy: 0.8921 - loss: 0.3020 390/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 778us/step - accuracy: 0.8917 - loss: 0.3036 453/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 781us/step - accuracy: 0.8914 - loss: 0.3047 517/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 781us/step - accuracy: 0.8912 - loss: 0.3056 581/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 782us/step - accuracy: 0.8910 - loss: 0.3062 647/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 781us/step - accuracy: 0.8909 - loss: 0.3064 710/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 783us/step - accuracy: 0.8909 - loss: 0.3065 774/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 783us/step - accuracy: 0.8908 - loss: 0.3067 840/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 782us/step - accuracy: 0.8908 - loss: 0.3069 905/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 782us/step - accuracy: 0.8907 - loss: 0.3072 971/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 780us/step - accuracy: 0.8907 - loss: 0.30721038/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 778us/step - accuracy: 0.8908 - loss: 0.30721104/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 777us/step - accuracy: 0.8908 - loss: 0.30711168/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 778us/step - accuracy: 0.8909 - loss: 0.30701232/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 778us/step - accuracy: 0.8909 - loss: 0.30701298/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 778us/step - accuracy: 0.8910 - loss: 0.30691365/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 777us/step - accuracy: 0.8911 - loss: 0.30681433/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 775us/step - accuracy: 0.8911 - loss: 0.30671500/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 774us/step - accuracy: 0.8912 - loss: 0.30661565/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 774us/step - accuracy: 0.8912 - loss: 0.30651631/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 774us/step - accuracy: 0.8913 - loss: 0.30641697/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 773us/step - accuracy: 0.8913 - loss: 0.30641719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 830us/step - accuracy: 0.8913 - loss: 0.3063 - val_accuracy: 0.8688 - val_loss: 0.3543
Epoch 15/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 18s 11ms/step - accuracy: 0.9375 - loss: 0.2799  67/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 766us/step - accuracy: 0.9028 - loss: 0.2701 130/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 783us/step - accuracy: 0.8986 - loss: 0.2814 194/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 783us/step - accuracy: 0.8958 - loss: 0.2895 260/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 779us/step - accuracy: 0.8948 - loss: 0.2925 324/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 779us/step - accuracy: 0.8942 - loss: 0.2943 384/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 790us/step - accuracy: 0.8939 - loss: 0.2958 445/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 794us/step - accuracy: 0.8936 - loss: 0.2970 506/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 799us/step - accuracy: 0.8934 - loss: 0.2979 567/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 801us/step - accuracy: 0.8932 - loss: 0.2986 630/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 801us/step - accuracy: 0.8931 - loss: 0.2989 691/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 803us/step - accuracy: 0.8930 - loss: 0.2991 754/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 803us/step - accuracy: 0.8929 - loss: 0.2993 815/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 805us/step - accuracy: 0.8929 - loss: 0.2995 876/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 806us/step - accuracy: 0.8928 - loss: 0.2998 937/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 807us/step - accuracy: 0.8928 - loss: 0.2999 997/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 809us/step - accuracy: 0.8929 - loss: 0.30001060/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 809us/step - accuracy: 0.8929 - loss: 0.29991122/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 810us/step - accuracy: 0.8930 - loss: 0.29991184/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 810us/step - accuracy: 0.8930 - loss: 0.29981247/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 810us/step - accuracy: 0.8930 - loss: 0.29981308/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 811us/step - accuracy: 0.8931 - loss: 0.29971371/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 810us/step - accuracy: 0.8932 - loss: 0.29961434/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 810us/step - accuracy: 0.8932 - loss: 0.29951495/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 810us/step - accuracy: 0.8933 - loss: 0.29951557/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 810us/step - accuracy: 0.8933 - loss: 0.29941619/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 811us/step - accuracy: 0.8934 - loss: 0.29931682/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 811us/step - accuracy: 0.8934 - loss: 0.29931719/1719 ━━━━━━━━━━━━━━━━━━━━ 2s 871us/step - accuracy: 0.8934 - loss: 0.2992 - val_accuracy: 0.8722 - val_loss: 0.3540
Epoch 16/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 20s 12ms/step - accuracy: 0.9375 - loss: 0.2743  61/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 835us/step - accuracy: 0.9037 - loss: 0.2624 123/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 826us/step - accuracy: 0.8997 - loss: 0.2731 186/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 819us/step - accuracy: 0.8968 - loss: 0.2821 250/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 812us/step - accuracy: 0.8957 - loss: 0.2854 312/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 811us/step - accuracy: 0.8952 - loss: 0.2873 376/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 807us/step - accuracy: 0.8950 - loss: 0.2888 439/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 807us/step - accuracy: 0.8948 - loss: 0.2902 500/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 809us/step - accuracy: 0.8947 - loss: 0.2912 560/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 813us/step - accuracy: 0.8946 - loss: 0.2919 619/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 816us/step - accuracy: 0.8945 - loss: 0.2923 679/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 818us/step - accuracy: 0.8945 - loss: 0.2925 739/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 820us/step - accuracy: 0.8945 - loss: 0.2927 800/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 820us/step - accuracy: 0.8945 - loss: 0.2929 858/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 824us/step - accuracy: 0.8944 - loss: 0.2931 915/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 828us/step - accuracy: 0.8944 - loss: 0.2934 971/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 832us/step - accuracy: 0.8944 - loss: 0.29351028/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 835us/step - accuracy: 0.8945 - loss: 0.29351085/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 838us/step - accuracy: 0.8945 - loss: 0.29341141/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 841us/step - accuracy: 0.8946 - loss: 0.29341199/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 842us/step - accuracy: 0.8946 - loss: 0.29331260/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 842us/step - accuracy: 0.8946 - loss: 0.29331319/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 843us/step - accuracy: 0.8947 - loss: 0.29331378/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 843us/step - accuracy: 0.8947 - loss: 0.29321438/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 843us/step - accuracy: 0.8948 - loss: 0.29311498/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 843us/step - accuracy: 0.8949 - loss: 0.29301557/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 843us/step - accuracy: 0.8949 - loss: 0.29301616/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 843us/step - accuracy: 0.8950 - loss: 0.29291675/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 843us/step - accuracy: 0.8950 - loss: 0.29291719/1719 ━━━━━━━━━━━━━━━━━━━━ 2s 907us/step - accuracy: 0.8950 - loss: 0.2928 - val_accuracy: 0.8730 - val_loss: 0.3506
Epoch 17/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 20s 12ms/step - accuracy: 0.9375 - loss: 0.2607  60/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 848us/step - accuracy: 0.9058 - loss: 0.2554 121/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 838us/step - accuracy: 0.9016 - loss: 0.2659 182/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 835us/step - accuracy: 0.8985 - loss: 0.2752 243/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 835us/step - accuracy: 0.8974 - loss: 0.2786 305/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 831us/step - accuracy: 0.8969 - loss: 0.2806 367/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 828us/step - accuracy: 0.8968 - loss: 0.2821 428/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 827us/step - accuracy: 0.8967 - loss: 0.2836 490/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 825us/step - accuracy: 0.8966 - loss: 0.2846 552/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 823us/step - accuracy: 0.8966 - loss: 0.2855 613/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 823us/step - accuracy: 0.8966 - loss: 0.2859 675/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 822us/step - accuracy: 0.8966 - loss: 0.2861 732/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 827us/step - accuracy: 0.8966 - loss: 0.2863 791/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 829us/step - accuracy: 0.8966 - loss: 0.2865 850/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 831us/step - accuracy: 0.8966 - loss: 0.2868 908/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 833us/step - accuracy: 0.8966 - loss: 0.2871 967/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 834us/step - accuracy: 0.8966 - loss: 0.28721024/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 837us/step - accuracy: 0.8967 - loss: 0.28721077/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 843us/step - accuracy: 0.8968 - loss: 0.28721110/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 863us/step - accuracy: 0.8968 - loss: 0.28711167/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 864us/step - accuracy: 0.8969 - loss: 0.28711221/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 867us/step - accuracy: 0.8969 - loss: 0.28711281/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 866us/step - accuracy: 0.8970 - loss: 0.28711337/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 867us/step - accuracy: 0.8970 - loss: 0.28701397/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 866us/step - accuracy: 0.8971 - loss: 0.28691458/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 865us/step - accuracy: 0.8972 - loss: 0.28691519/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 863us/step - accuracy: 0.8972 - loss: 0.28681581/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 861us/step - accuracy: 0.8973 - loss: 0.28671638/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 862us/step - accuracy: 0.8973 - loss: 0.28671698/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 861us/step - accuracy: 0.8973 - loss: 0.28671719/1719 ━━━━━━━━━━━━━━━━━━━━ 2s 922us/step - accuracy: 0.8973 - loss: 0.2867 - val_accuracy: 0.8744 - val_loss: 0.3491
Epoch 18/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 19s 12ms/step - accuracy: 0.9688 - loss: 0.2506  62/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 821us/step - accuracy: 0.9096 - loss: 0.2494 123/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 823us/step - accuracy: 0.9050 - loss: 0.2600 183/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 830us/step - accuracy: 0.9021 - loss: 0.2689 240/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 844us/step - accuracy: 0.9010 - loss: 0.2722 299/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 846us/step - accuracy: 0.9004 - loss: 0.2742 359/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 846us/step - accuracy: 0.9001 - loss: 0.2757 419/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 846us/step - accuracy: 0.8999 - loss: 0.2773 479/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 846us/step - accuracy: 0.8998 - loss: 0.2783 541/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 843us/step - accuracy: 0.8996 - loss: 0.2792 603/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 839us/step - accuracy: 0.8996 - loss: 0.2797 666/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 835us/step - accuracy: 0.8996 - loss: 0.2800 729/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 832us/step - accuracy: 0.8996 - loss: 0.2802 792/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 830us/step - accuracy: 0.8996 - loss: 0.2805 857/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 826us/step - accuracy: 0.8996 - loss: 0.2808 922/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 822us/step - accuracy: 0.8995 - loss: 0.2811 987/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 819us/step - accuracy: 0.8996 - loss: 0.28121050/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 818us/step - accuracy: 0.8996 - loss: 0.28121115/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 815us/step - accuracy: 0.8997 - loss: 0.28111179/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 813us/step - accuracy: 0.8997 - loss: 0.28111242/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 813us/step - accuracy: 0.8998 - loss: 0.28111304/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 813us/step - accuracy: 0.8998 - loss: 0.28111363/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 815us/step - accuracy: 0.8998 - loss: 0.28101421/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 817us/step - accuracy: 0.8999 - loss: 0.28091481/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 818us/step - accuracy: 0.8999 - loss: 0.28091540/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 819us/step - accuracy: 0.9000 - loss: 0.28081600/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 820us/step - accuracy: 0.9000 - loss: 0.28081659/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 821us/step - accuracy: 0.9000 - loss: 0.28081719/1719 ━━━━━━━━━━━━━━━━━━━━ 2s 880us/step - accuracy: 0.9000 - loss: 0.2807 - val_accuracy: 0.8750 - val_loss: 0.3471
Epoch 19/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 19s 11ms/step - accuracy: 0.9688 - loss: 0.2360  63/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 819us/step - accuracy: 0.9115 - loss: 0.2432 128/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 798us/step - accuracy: 0.9067 - loss: 0.2549 191/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 797us/step - accuracy: 0.9040 - loss: 0.2635 255/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 794us/step - accuracy: 0.9030 - loss: 0.2669 319/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 793us/step - accuracy: 0.9025 - loss: 0.2689 381/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 796us/step - accuracy: 0.9022 - loss: 0.2706 445/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 794us/step - accuracy: 0.9020 - loss: 0.2721 510/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.9019 - loss: 0.2732 576/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 788us/step - accuracy: 0.9018 - loss: 0.2739 642/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 786us/step - accuracy: 0.9018 - loss: 0.2743 705/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 787us/step - accuracy: 0.9018 - loss: 0.2745 769/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 787us/step - accuracy: 0.9017 - loss: 0.2748 832/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 788us/step - accuracy: 0.9017 - loss: 0.2750 896/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 788us/step - accuracy: 0.9017 - loss: 0.2754 960/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 788us/step - accuracy: 0.9017 - loss: 0.27551024/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 788us/step - accuracy: 0.9017 - loss: 0.27561086/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 789us/step - accuracy: 0.9017 - loss: 0.27551150/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 789us/step - accuracy: 0.9017 - loss: 0.27551210/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.9018 - loss: 0.27551271/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 793us/step - accuracy: 0.9018 - loss: 0.27551332/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 795us/step - accuracy: 0.9018 - loss: 0.27541391/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 797us/step - accuracy: 0.9018 - loss: 0.27541452/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 798us/step - accuracy: 0.9019 - loss: 0.27531515/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 798us/step - accuracy: 0.9019 - loss: 0.27531578/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 798us/step - accuracy: 0.9019 - loss: 0.27521638/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 800us/step - accuracy: 0.9020 - loss: 0.27521697/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 802us/step - accuracy: 0.9020 - loss: 0.27521719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 865us/step - accuracy: 0.9020 - loss: 0.2751 - val_accuracy: 0.8752 - val_loss: 0.3461
Epoch 20/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 20s 12ms/step - accuracy: 0.9688 - loss: 0.2272  62/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 830us/step - accuracy: 0.9126 - loss: 0.2383 123/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 825us/step - accuracy: 0.9084 - loss: 0.2488 185/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 819us/step - accuracy: 0.9060 - loss: 0.2580 250/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 809us/step - accuracy: 0.9050 - loss: 0.2616 314/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 805us/step - accuracy: 0.9046 - loss: 0.2636 377/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 805us/step - accuracy: 0.9044 - loss: 0.2652 442/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 800us/step - accuracy: 0.9042 - loss: 0.2668 505/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 800us/step - accuracy: 0.9040 - loss: 0.2679 570/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 798us/step - accuracy: 0.9039 - loss: 0.2686 633/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 798us/step - accuracy: 0.9038 - loss: 0.2690 698/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 796us/step - accuracy: 0.9038 - loss: 0.2692 763/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 794us/step - accuracy: 0.9037 - loss: 0.2695 827/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 794us/step - accuracy: 0.9037 - loss: 0.2697 892/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.9036 - loss: 0.2701 956/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.9036 - loss: 0.27021018/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 793us/step - accuracy: 0.9036 - loss: 0.27031081/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 793us/step - accuracy: 0.9036 - loss: 0.27031145/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 793us/step - accuracy: 0.9036 - loss: 0.27021209/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 793us/step - accuracy: 0.9036 - loss: 0.27021273/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.9036 - loss: 0.27021335/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 793us/step - accuracy: 0.9036 - loss: 0.27011388/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 799us/step - accuracy: 0.9037 - loss: 0.27011443/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 804us/step - accuracy: 0.9037 - loss: 0.27001503/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 805us/step - accuracy: 0.9037 - loss: 0.27001565/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 806us/step - accuracy: 0.9037 - loss: 0.26991626/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 806us/step - accuracy: 0.9037 - loss: 0.26991687/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 807us/step - accuracy: 0.9037 - loss: 0.26991719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 866us/step - accuracy: 0.9037 - loss: 0.2699 - val_accuracy: 0.8754 - val_loss: 0.3451
Epoch 21/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 19s 11ms/step - accuracy: 0.9688 - loss: 0.2231  63/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 808us/step - accuracy: 0.9150 - loss: 0.2335 123/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 821us/step - accuracy: 0.9111 - loss: 0.2436 183/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 826us/step - accuracy: 0.9088 - loss: 0.2526 243/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 828us/step - accuracy: 0.9079 - loss: 0.2561 304/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 828us/step - accuracy: 0.9074 - loss: 0.2582 363/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 833us/step - accuracy: 0.9071 - loss: 0.2597 423/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 833us/step - accuracy: 0.9068 - loss: 0.2613 484/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 832us/step - accuracy: 0.9066 - loss: 0.2624 544/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 833us/step - accuracy: 0.9064 - loss: 0.2633 607/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 829us/step - accuracy: 0.9062 - loss: 0.2637 670/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 827us/step - accuracy: 0.9062 - loss: 0.2640 732/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 826us/step - accuracy: 0.9061 - loss: 0.2642 795/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 824us/step - accuracy: 0.9060 - loss: 0.2645 857/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 823us/step - accuracy: 0.9059 - loss: 0.2648 919/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 822us/step - accuracy: 0.9058 - loss: 0.2650 981/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 821us/step - accuracy: 0.9058 - loss: 0.26511044/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 820us/step - accuracy: 0.9058 - loss: 0.26511106/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 819us/step - accuracy: 0.9058 - loss: 0.26511169/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 818us/step - accuracy: 0.9058 - loss: 0.26511232/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 818us/step - accuracy: 0.9058 - loss: 0.26511293/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 818us/step - accuracy: 0.9058 - loss: 0.26511353/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 819us/step - accuracy: 0.9058 - loss: 0.26501412/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 821us/step - accuracy: 0.9059 - loss: 0.26501471/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 822us/step - accuracy: 0.9059 - loss: 0.26491530/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 823us/step - accuracy: 0.9059 - loss: 0.26491590/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 824us/step - accuracy: 0.9059 - loss: 0.26481651/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 824us/step - accuracy: 0.9059 - loss: 0.26481716/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 822us/step - accuracy: 0.9059 - loss: 0.26481719/1719 ━━━━━━━━━━━━━━━━━━━━ 2s 881us/step - accuracy: 0.9059 - loss: 0.2648 - val_accuracy: 0.8768 - val_loss: 0.3438
Epoch 22/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 19s 11ms/step - accuracy: 0.9688 - loss: 0.2181  64/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 794us/step - accuracy: 0.9141 - loss: 0.2291 129/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 785us/step - accuracy: 0.9109 - loss: 0.2401 191/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 792us/step - accuracy: 0.9090 - loss: 0.2484 250/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 808us/step - accuracy: 0.9083 - loss: 0.2516 310/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 813us/step - accuracy: 0.9079 - loss: 0.2535 371/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 816us/step - accuracy: 0.9078 - loss: 0.2551 432/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 818us/step - accuracy: 0.9075 - loss: 0.2567 492/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 820us/step - accuracy: 0.9074 - loss: 0.2578 553/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 821us/step - accuracy: 0.9072 - loss: 0.2586 615/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 820us/step - accuracy: 0.9072 - loss: 0.2591 677/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 819us/step - accuracy: 0.9072 - loss: 0.2593 741/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 817us/step - accuracy: 0.9071 - loss: 0.2595 807/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 813us/step - accuracy: 0.9071 - loss: 0.2598 870/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 812us/step - accuracy: 0.9070 - loss: 0.2601 935/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 810us/step - accuracy: 0.9070 - loss: 0.2603 999/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 808us/step - accuracy: 0.9070 - loss: 0.26041063/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 807us/step - accuracy: 0.9070 - loss: 0.26041127/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 805us/step - accuracy: 0.9070 - loss: 0.26041192/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 804us/step - accuracy: 0.9070 - loss: 0.26031256/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 803us/step - accuracy: 0.9070 - loss: 0.26031321/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 801us/step - accuracy: 0.9071 - loss: 0.26031387/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 800us/step - accuracy: 0.9071 - loss: 0.26031451/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 799us/step - accuracy: 0.9072 - loss: 0.26021517/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 798us/step - accuracy: 0.9072 - loss: 0.26011583/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 796us/step - accuracy: 0.9072 - loss: 0.26011647/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 796us/step - accuracy: 0.9072 - loss: 0.26011712/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 795us/step - accuracy: 0.9072 - loss: 0.26011719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 852us/step - accuracy: 0.9072 - loss: 0.2601 - val_accuracy: 0.8756 - val_loss: 0.3455
Epoch 23/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 18s 11ms/step - accuracy: 0.9375 - loss: 0.2152  65/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 789us/step - accuracy: 0.9135 - loss: 0.2245 129/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 785us/step - accuracy: 0.9114 - loss: 0.2352 195/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 779us/step - accuracy: 0.9097 - loss: 0.2437 260/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 777us/step - accuracy: 0.9091 - loss: 0.2469 324/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 778us/step - accuracy: 0.9088 - loss: 0.2488 389/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 779us/step - accuracy: 0.9087 - loss: 0.2507 451/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 783us/step - accuracy: 0.9086 - loss: 0.2521 516/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 781us/step - accuracy: 0.9085 - loss: 0.2532 581/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 781us/step - accuracy: 0.9085 - loss: 0.2539 646/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 781us/step - accuracy: 0.9085 - loss: 0.2542 711/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 781us/step - accuracy: 0.9085 - loss: 0.2545 775/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 781us/step - accuracy: 0.9085 - loss: 0.2547 838/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 783us/step - accuracy: 0.9084 - loss: 0.2550 900/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 785us/step - accuracy: 0.9084 - loss: 0.2554 965/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 784us/step - accuracy: 0.9084 - loss: 0.25551030/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 783us/step - accuracy: 0.9084 - loss: 0.25551095/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 783us/step - accuracy: 0.9084 - loss: 0.25551160/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 782us/step - accuracy: 0.9084 - loss: 0.25551223/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 784us/step - accuracy: 0.9084 - loss: 0.25551283/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 786us/step - accuracy: 0.9085 - loss: 0.25551345/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 788us/step - accuracy: 0.9085 - loss: 0.25541407/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 789us/step - accuracy: 0.9085 - loss: 0.25541469/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 790us/step - accuracy: 0.9086 - loss: 0.25531532/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 791us/step - accuracy: 0.9086 - loss: 0.25531593/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.9086 - loss: 0.25531656/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 792us/step - accuracy: 0.9086 - loss: 0.25521719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 850us/step - accuracy: 0.9086 - loss: 0.2552 - val_accuracy: 0.8760 - val_loss: 0.3454
Epoch 24/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 18s 11ms/step - accuracy: 0.9375 - loss: 0.2105  63/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 810us/step - accuracy: 0.9155 - loss: 0.2194 126/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 805us/step - accuracy: 0.9141 - loss: 0.2297 190/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 800us/step - accuracy: 0.9125 - loss: 0.2384 252/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 803us/step - accuracy: 0.9119 - loss: 0.2418 316/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 800us/step - accuracy: 0.9115 - loss: 0.2438 381/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 795us/step - accuracy: 0.9113 - loss: 0.2456 445/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 795us/step - accuracy: 0.9111 - loss: 0.2472 508/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 795us/step - accuracy: 0.9110 - loss: 0.2483 568/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 800us/step - accuracy: 0.9109 - loss: 0.2490 628/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 804us/step - accuracy: 0.9109 - loss: 0.2494 688/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 807us/step - accuracy: 0.9108 - loss: 0.2497 749/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 809us/step - accuracy: 0.9108 - loss: 0.2499 810/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 811us/step - accuracy: 0.9108 - loss: 0.2502 871/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 811us/step - accuracy: 0.9107 - loss: 0.2505 934/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 811us/step - accuracy: 0.9106 - loss: 0.2507 994/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 813us/step - accuracy: 0.9106 - loss: 0.25081056/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 812us/step - accuracy: 0.9106 - loss: 0.25081119/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 812us/step - accuracy: 0.9106 - loss: 0.25081182/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 811us/step - accuracy: 0.9106 - loss: 0.25081247/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 809us/step - accuracy: 0.9106 - loss: 0.25081309/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 809us/step - accuracy: 0.9106 - loss: 0.25071373/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 808us/step - accuracy: 0.9107 - loss: 0.25071438/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 807us/step - accuracy: 0.9107 - loss: 0.25061503/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 806us/step - accuracy: 0.9107 - loss: 0.25061567/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 805us/step - accuracy: 0.9107 - loss: 0.25061631/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 804us/step - accuracy: 0.9107 - loss: 0.25061695/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 803us/step - accuracy: 0.9107 - loss: 0.25061719/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 860us/step - accuracy: 0.9107 - loss: 0.2506 - val_accuracy: 0.8752 - val_loss: 0.3457
Epoch 25/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 19s 11ms/step - accuracy: 0.9375 - loss: 0.2023  66/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 773us/step - accuracy: 0.9190 - loss: 0.2148 128/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 793us/step - accuracy: 0.9168 - loss: 0.2252 189/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 802us/step - accuracy: 0.9150 - loss: 0.2335 250/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 809us/step - accuracy: 0.9143 - loss: 0.2370 311/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 813us/step - accuracy: 0.9139 - loss: 0.2390 371/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 818us/step - accuracy: 0.9138 - loss: 0.2407 431/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 821us/step - accuracy: 0.9136 - loss: 0.2423 492/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 822us/step - accuracy: 0.9135 - loss: 0.2435 554/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 821us/step - accuracy: 0.9133 - loss: 0.2444 614/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 823us/step - accuracy: 0.9133 - loss: 0.2448 674/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 824us/step - accuracy: 0.9132 - loss: 0.2451 734/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 826us/step - accuracy: 0.9132 - loss: 0.2453 795/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 826us/step - accuracy: 0.9131 - loss: 0.2456 857/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 825us/step - accuracy: 0.9131 - loss: 0.2459 919/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 824us/step - accuracy: 0.9130 - loss: 0.2462 980/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 824us/step - accuracy: 0.9129 - loss: 0.24631040/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 825us/step - accuracy: 0.9129 - loss: 0.24631099/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 827us/step - accuracy: 0.9129 - loss: 0.24631157/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 829us/step - accuracy: 0.9129 - loss: 0.24631215/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 831us/step - accuracy: 0.9129 - loss: 0.24631273/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 833us/step - accuracy: 0.9128 - loss: 0.24631332/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 834us/step - accuracy: 0.9128 - loss: 0.24631386/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 838us/step - accuracy: 0.9129 - loss: 0.24621439/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 842us/step - accuracy: 0.9129 - loss: 0.24621492/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 845us/step - accuracy: 0.9129 - loss: 0.24621548/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 847us/step - accuracy: 0.9128 - loss: 0.24611604/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 849us/step - accuracy: 0.9128 - loss: 0.24611660/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 851us/step - accuracy: 0.9128 - loss: 0.24611718/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 852us/step - accuracy: 0.9128 - loss: 0.24611719/1719 ━━━━━━━━━━━━━━━━━━━━ 2s 916us/step - accuracy: 0.9128 - loss: 0.2461 - val_accuracy: 0.8758 - val_loss: 0.3452
Epoch 26/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 20s 12ms/step - accuracy: 0.9375 - loss: 0.2015  60/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 859us/step - accuracy: 0.9229 - loss: 0.2100 119/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 859us/step - accuracy: 0.9199 - loss: 0.2190 178/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 856us/step - accuracy: 0.9174 - loss: 0.2283 240/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 846us/step - accuracy: 0.9163 - loss: 0.2322 303/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 836us/step - accuracy: 0.9158 - loss: 0.2344 363/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 837us/step - accuracy: 0.9155 - loss: 0.2361 422/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 840us/step - accuracy: 0.9151 - loss: 0.2377 481/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 842us/step - accuracy: 0.9149 - loss: 0.2389 542/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 841us/step - accuracy: 0.9147 - loss: 0.2399 601/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 841us/step - accuracy: 0.9146 - loss: 0.2404 660/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 842us/step - accuracy: 0.9145 - loss: 0.2407 722/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 839us/step - accuracy: 0.9145 - loss: 0.2409 782/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 839us/step - accuracy: 0.9144 - loss: 0.2412 842/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 839us/step - accuracy: 0.9143 - loss: 0.2415 880/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 860us/step - accuracy: 0.9143 - loss: 0.2417 938/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 860us/step - accuracy: 0.9142 - loss: 0.2419 997/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 860us/step - accuracy: 0.9141 - loss: 0.24201054/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 861us/step - accuracy: 0.9141 - loss: 0.24211112/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 862us/step - accuracy: 0.9141 - loss: 0.24201166/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 865us/step - accuracy: 0.9141 - loss: 0.24201217/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 870us/step - accuracy: 0.9141 - loss: 0.24201273/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 872us/step - accuracy: 0.9140 - loss: 0.24201326/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 875us/step - accuracy: 0.9140 - loss: 0.24201385/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 874us/step - accuracy: 0.9140 - loss: 0.24201445/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 873us/step - accuracy: 0.9141 - loss: 0.24191507/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 871us/step - accuracy: 0.9140 - loss: 0.24191567/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 870us/step - accuracy: 0.9140 - loss: 0.24191625/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 870us/step - accuracy: 0.9140 - loss: 0.24191687/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 868us/step - accuracy: 0.9140 - loss: 0.24191719/1719 ━━━━━━━━━━━━━━━━━━━━ 2s 929us/step - accuracy: 0.9140 - loss: 0.2419 - val_accuracy: 0.8748 - val_loss: 0.3459
Epoch 27/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 20s 12ms/step - accuracy: 0.9375 - loss: 0.1958  62/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 832us/step - accuracy: 0.9247 - loss: 0.2064 124/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 821us/step - accuracy: 0.9218 - loss: 0.2162 186/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 820us/step - accuracy: 0.9194 - loss: 0.2250 243/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 835us/step - accuracy: 0.9184 - loss: 0.2284 300/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 844us/step - accuracy: 0.9179 - loss: 0.2304 361/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 843us/step - accuracy: 0.9176 - loss: 0.2321 421/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 843us/step - accuracy: 0.9173 - loss: 0.2337 480/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 845us/step - accuracy: 0.9170 - loss: 0.2349 537/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 849us/step - accuracy: 0.9168 - loss: 0.2358 597/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 849us/step - accuracy: 0.9167 - loss: 0.2363 654/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 852us/step - accuracy: 0.9166 - loss: 0.2366 712/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 853us/step - accuracy: 0.9165 - loss: 0.2368 770/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 855us/step - accuracy: 0.9164 - loss: 0.2371 828/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 856us/step - accuracy: 0.9164 - loss: 0.2373 885/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 858us/step - accuracy: 0.9162 - loss: 0.2377 922/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 878us/step - accuracy: 0.9162 - loss: 0.2378 979/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 879us/step - accuracy: 0.9161 - loss: 0.23791036/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 879us/step - accuracy: 0.9161 - loss: 0.23791095/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 878us/step - accuracy: 0.9160 - loss: 0.23791153/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 877us/step - accuracy: 0.9160 - loss: 0.23791212/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 876us/step - accuracy: 0.9160 - loss: 0.23791272/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 875us/step - accuracy: 0.9159 - loss: 0.23791331/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 874us/step - accuracy: 0.9159 - loss: 0.23791391/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 872us/step - accuracy: 0.9159 - loss: 0.23781456/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 868us/step - accuracy: 0.9159 - loss: 0.23781520/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 865us/step - accuracy: 0.9159 - loss: 0.23781584/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 862us/step - accuracy: 0.9159 - loss: 0.23781649/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 858us/step - accuracy: 0.9159 - loss: 0.23771713/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 856us/step - accuracy: 0.9159 - loss: 0.23771719/1719 ━━━━━━━━━━━━━━━━━━━━ 2s 913us/step - accuracy: 0.9159 - loss: 0.2377 - val_accuracy: 0.8756 - val_loss: 0.3461
Epoch 28/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 20s 12ms/step - accuracy: 0.9375 - loss: 0.1900  63/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 808us/step - accuracy: 0.9282 - loss: 0.2021 127/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 797us/step - accuracy: 0.9244 - loss: 0.2123 189/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 801us/step - accuracy: 0.9219 - loss: 0.2208 252/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 800us/step - accuracy: 0.9207 - loss: 0.2244 314/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 802us/step - accuracy: 0.9201 - loss: 0.2264 378/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 800us/step - accuracy: 0.9197 - loss: 0.2282 441/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 800us/step - accuracy: 0.9192 - loss: 0.2298 505/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 799us/step - accuracy: 0.9189 - loss: 0.2310 569/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 798us/step - accuracy: 0.9186 - loss: 0.2318 631/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 799us/step - accuracy: 0.9184 - loss: 0.2322 695/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 798us/step - accuracy: 0.9183 - loss: 0.2325 750/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 806us/step - accuracy: 0.9182 - loss: 0.2327 809/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 810us/step - accuracy: 0.9181 - loss: 0.2330 869/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 811us/step - accuracy: 0.9180 - loss: 0.2333 931/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 811us/step - accuracy: 0.9178 - loss: 0.2335 995/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 809us/step - accuracy: 0.9177 - loss: 0.23361058/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 809us/step - accuracy: 0.9177 - loss: 0.23371121/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 808us/step - accuracy: 0.9176 - loss: 0.23371184/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 808us/step - accuracy: 0.9176 - loss: 0.23361248/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 807us/step - accuracy: 0.9175 - loss: 0.23361313/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 805us/step - accuracy: 0.9175 - loss: 0.23361377/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 804us/step - accuracy: 0.9175 - loss: 0.23361438/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 805us/step - accuracy: 0.9175 - loss: 0.23361496/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 808us/step - accuracy: 0.9175 - loss: 0.23351553/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 811us/step - accuracy: 0.9174 - loss: 0.23351611/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 813us/step - accuracy: 0.9174 - loss: 0.23351668/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 815us/step - accuracy: 0.9174 - loss: 0.23351719/1719 ━━━━━━━━━━━━━━━━━━━━ 2s 877us/step - accuracy: 0.9174 - loss: 0.2335 - val_accuracy: 0.8752 - val_loss: 0.3471
Epoch 29/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 21s 12ms/step - accuracy: 0.9375 - loss: 0.1863  61/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 844us/step - accuracy: 0.9292 - loss: 0.1982 123/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 832us/step - accuracy: 0.9263 - loss: 0.2075 184/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 830us/step - accuracy: 0.9238 - loss: 0.2162 246/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 825us/step - accuracy: 0.9226 - loss: 0.2199 309/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 820us/step - accuracy: 0.9217 - loss: 0.2221 371/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 819us/step - accuracy: 0.9212 - loss: 0.2239 433/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 818us/step - accuracy: 0.9207 - loss: 0.2256 497/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 814us/step - accuracy: 0.9203 - loss: 0.2268 559/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 814us/step - accuracy: 0.9200 - loss: 0.2277 621/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 814us/step - accuracy: 0.9198 - loss: 0.2281 684/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 812us/step - accuracy: 0.9197 - loss: 0.2284 747/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 811us/step - accuracy: 0.9196 - loss: 0.2287 810/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 810us/step - accuracy: 0.9195 - loss: 0.2290 872/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 810us/step - accuracy: 0.9193 - loss: 0.2294 934/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 810us/step - accuracy: 0.9192 - loss: 0.2296 995/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 811us/step - accuracy: 0.9191 - loss: 0.22971058/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 810us/step - accuracy: 0.9191 - loss: 0.22971121/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 809us/step - accuracy: 0.9190 - loss: 0.22971184/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 809us/step - accuracy: 0.9190 - loss: 0.22971245/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 810us/step - accuracy: 0.9189 - loss: 0.22971300/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 814us/step - accuracy: 0.9189 - loss: 0.22971356/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 818us/step - accuracy: 0.9189 - loss: 0.22971416/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 819us/step - accuracy: 0.9189 - loss: 0.22961477/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 819us/step - accuracy: 0.9189 - loss: 0.22961540/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 818us/step - accuracy: 0.9189 - loss: 0.22961602/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 817us/step - accuracy: 0.9189 - loss: 0.22961665/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 817us/step - accuracy: 0.9188 - loss: 0.22961719/1719 ━━━━━━━━━━━━━━━━━━━━ 2s 874us/step - accuracy: 0.9188 - loss: 0.2296 - val_accuracy: 0.8756 - val_loss: 0.3481
Epoch 30/30
   1/1719 ━━━━━━━━━━━━━━━━━━━━ 19s 11ms/step - accuracy: 0.9375 - loss: 0.1867  64/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 794us/step - accuracy: 0.9295 - loss: 0.1950 127/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 799us/step - accuracy: 0.9263 - loss: 0.2046 184/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 824us/step - accuracy: 0.9242 - loss: 0.2125 239/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 846us/step - accuracy: 0.9234 - loss: 0.2159 295/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 856us/step - accuracy: 0.9228 - loss: 0.2180 353/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 859us/step - accuracy: 0.9225 - loss: 0.2196 410/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 863us/step - accuracy: 0.9220 - loss: 0.2213 468/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 864us/step - accuracy: 0.9216 - loss: 0.2226 531/1719 ━━━━━━━━━━━━━━━━━━━━ 1s 856us/step - accuracy: 0.9213 - loss: 0.2237 593/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 852us/step - accuracy: 0.9211 - loss: 0.2243 656/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 847us/step - accuracy: 0.9210 - loss: 0.2246 718/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 845us/step - accuracy: 0.9208 - loss: 0.2249 781/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 841us/step - accuracy: 0.9207 - loss: 0.2252 844/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 838us/step - accuracy: 0.9206 - loss: 0.2255 906/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 836us/step - accuracy: 0.9205 - loss: 0.2258 969/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 833us/step - accuracy: 0.9204 - loss: 0.22601033/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 830us/step - accuracy: 0.9203 - loss: 0.22601091/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 832us/step - accuracy: 0.9202 - loss: 0.22601148/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 835us/step - accuracy: 0.9202 - loss: 0.22601205/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 837us/step - accuracy: 0.9201 - loss: 0.22601261/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 840us/step - accuracy: 0.9201 - loss: 0.22601316/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 843us/step - accuracy: 0.9201 - loss: 0.22601372/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 845us/step - accuracy: 0.9201 - loss: 0.22591429/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 847us/step - accuracy: 0.9201 - loss: 0.22591487/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 848us/step - accuracy: 0.9201 - loss: 0.22591543/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 850us/step - accuracy: 0.9201 - loss: 0.22591599/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 852us/step - accuracy: 0.9200 - loss: 0.22581656/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 853us/step - accuracy: 0.9200 - loss: 0.22581714/1719 ━━━━━━━━━━━━━━━━━━━━ 0s 853us/step - accuracy: 0.9200 - loss: 0.22591719/1719 ━━━━━━━━━━━━━━━━━━━━ 2s 913us/step - accuracy: 0.9200 - loss: 0.2259 - val_accuracy: 0.8774 - val_loss: 0.3476

The model is provided with both a taining set and a validation set. At each step, the model will report its performance on both sets. This will also allow to visualize the accuracy and loss curves on both sets (more later).

When calling the fit method in Keras (or similar frameworks), each step corresponds to the evaluation of a mini-batch. A mini-batch is a subset of the training data, and during each step, the model updates its weights based on the error calculated from this mini-batch.

An epoch is defined as one complete pass through the entire training dataset. During an epoch, the model processes multiple mini-batches until it has seen all the training data once. This process is repeated for a specified number of epochs to optimize the model’s performance.

Visualization

import pandas as pd 

pd.DataFrame(history.history).plot(
    figsize=(8, 5), xlim=[0, 29], ylim=[0, 1], grid=True, xlabel="Epoch",
    style=["r--", "r--.", "b-", "b-*"])
plt.legend(loc="lower left")  # extra code
plt.show()

Visualization

Evaluating the model on our test

model.evaluate(X_test, y_test)

  1/313 ━━━━━━━━━━━━━━━━━━━━ 3s 10ms/step - accuracy: 0.8750 - loss: 0.5315103/313 ━━━━━━━━━━━━━━━━━━━━ 0s 492us/step - accuracy: 0.8815 - loss: 0.3558209/313 ━━━━━━━━━━━━━━━━━━━━ 0s 483us/step - accuracy: 0.8766 - loss: 0.3650313/313 ━━━━━━━━━━━━━━━━━━━━ 0s 502us/step - accuracy: 0.8753 - loss: 0.3664

[0.3637523949146271, 0.8744000196456909]

Making predictions

X_new = X_test[:3]
y_proba = model.predict(X_new)
y_proba.round(2)

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 24ms/step1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 32ms/step

array([[0.  , 0.  , 0.  , 0.  , 0.  , 0.13, 0.  , 0.02, 0.  , 0.85],
       [0.  , 0.  , 1.  , 0.  , 0.  , 0.  , 0.  , 0.  , 0.  , 0.  ],
       [0.  , 1.  , 0.  , 0.  , 0.  , 0.  , 0.  , 0.  , 0.  , 0.  ]],
      dtype=float32)

y_pred = y_proba.argmax(axis=-1)
y_pred

array([9, 2, 1])

y_new = y_test[:3]
y_new

array([9, 2, 1], dtype=uint8)

As can be seen, the predictions are unambiguous, with only one class per prediction exhibiting a high value.

Predicted vs Observed

np.array(class_names)[y_pred]

array(['Ankle boot', 'Pullover', 'Trouser'], dtype='<U11')

Test Set Performance

from sklearn.metrics import classification_report

y_proba = model.predict(X_test)
y_pred = y_proba.argmax(axis=-1)

Test Set Performance

print(classification_report(y_test, y_pred))

              precision    recall  f1-score   support

           0       0.83      0.83      0.83      1000
           1       0.99      0.97      0.98      1000
           2       0.74      0.85      0.79      1000
           3       0.81      0.93      0.87      1000
           4       0.81      0.81      0.81      1000
           5       0.89      0.98      0.93      1000
           6       0.80      0.57      0.67      1000
           7       0.94      0.91      0.93      1000
           8       0.96      0.97      0.96      1000
           9       0.98      0.93      0.95      1000

    accuracy                           0.87     10000
   macro avg       0.88      0.87      0.87     10000
weighted avg       0.88      0.87      0.87     10000

Prologue

Summary

• Introduction to Neural Networks and Connectionism
  • Shift from symbolic AI to connectionist approaches in artificial intelligence.
  • Inspiration from biological neural networks and the human brain’s structure.
• Computations with Neurodes and Threshold Logic Units
  • Early models of neurons (neurodes) capable of performing logical operations (AND, OR, NOT).
  • Limitations of simple perceptrons in solving non-linearly separable problems like XOR.
• Multilayer Perceptrons (MLPs) and Feedforward Neural Networks (FNNs)
  • Overcoming perceptron limitations by introducing hidden layers.
  • Structure and information flow in feedforward neural networks.
  • Explanation of forward pass computations in neural networks.
• Activation Functions in Neural Networks
  • Importance of nonlinear activation functions (sigmoid, tanh, ReLU) for enabling learning of complex patterns.
  • Role of activation functions in backpropagation and gradient descent optimization.
  • Universal Approximation Theorem and its implications for neural networks.
• Deep Learning Frameworks
  • Overview of PyTorch and TensorFlow as leading platforms for deep learning.
  • Introduction to Keras as a high-level API for building and training neural networks.
  • Discussion on the suitability of different frameworks for research and industry applications.
• Hands-On Implementation with Keras
  • Loading and exploring the Fashion-MNIST dataset.
  • Building a neural network model using Keras’ Sequential API.
  • Compiling the model with appropriate loss functions and optimizers for multiclass classification.
  • Training the model and visualizing training and validation metrics over epochs.
  • Evaluating model performance on test data and interpreting results.
• Making Predictions and Interpreting Results
  • Using the trained model to make predictions on new data.
  • Visualizing predictions alongside actual images and labels.
  • Understanding the output probabilities and class assignments in the context of the dataset.

Next lecture

• We will discuss the training algorithm for artificial neural networks.

References

Cybenko, George V. 1989. “Approximation by Superpositions of a Sigmoidal Function.” Mathematics of Control, Signals and Systems 2: 303–14. https://api.semanticscholar.org/CorpusID:3958369.

Géron, Aurélien. 2022. Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow. 3rd ed. O’Reilly Media, Inc.

Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. Adaptive Computation and Machine Learning. MIT Press. https://dblp.org/rec/books/daglib/0040158.

Hornik, Kurt, Maxwell Stinchcombe, and Halbert White. 1989. “Multilayer Feedforward Networks Are Universal Approximators.” Neural Networks 2 (5): 359–66. https://doi.org/https://doi.org/10.1016/0893-6080(89)90020-8.

LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. 2015. “Deep Learning.” Nature 521 (7553): 436–44. https://doi.org/10.1038/nature14539.

LeNail, Alexander. 2019. “NN-SVG: Publication-Ready Neural Network Architecture Schematics.” Journal of Open Source Software 4 (33): 747. https://doi.org/10.21105/joss.00747.

McCulloch, Warren S, and Walter Pitts. 1943. “A logical calculus of the ideas immanent in nervous activity.” The Bulletin of Mathematical Biophysics 5 (4): 115–33. https://doi.org/10.1007/bf02478259.

Minsky, Marvin, and Seymour Papert. 1969. Perceptrons: An Introduction to Computational Geometry. Cambridge, MA, USA: MIT Press.

Rosenblatt, F. 1958. “The perceptron: A probabilistic model for information storage and organization in the brain.” Psychological Review 65 (6): 386–408. https://doi.org/10.1037/h0042519.

Russell, Stuart, and Peter Norvig. 2020. Artificial Intelligence: A Modern Approach. 4th ed. Pearson. http://aima.cs.berkeley.edu/.

Marcel Turcotte

Marcel.Turcotte@uOttawa.ca

School of Electrical Engineering and Computer Science (EECS)

University of Ottawa