TensorFlow.js Primer

Chapter 4

TensorFlow.js, like any other JavaScript library, follows the same workflow, as we will see in the next few sections. It is important to note, however, the flavors in which TensorFlow.js is available. A quick look at the TensorFlow.js help site (https://js.tensorflow.org/) can answer this question. As of the writing of this book, the following four flavors of TensorFlow.js were available for use:

1. TensorFlow.js (Client-side JavaScript development)

2. TensorFlow.js (Server-side development with Node.js)

3. tfjs-vis (Data Visualization)

4. tfjs-react-native (React Native JavaScript development)

This book is going to focus on client-side development using JavaScript with libraries i and iii above.

The TensorFlow.js Library

To understand development using a client-side library like TensorFlow.js, it is first important to understand the process of machine learning development in general and then apply that knowledge to development using the library.

Workflow

The flowchart in Figure 4-1 explains a generic workflow of machine learning using any machine learning library, whether client-side or server-side.

Figure 4-1: Machine Learning Workflow using TensorFlow.js

The value types for features in a dataset were first mentioned in Chapter 2 (see Figure 2-4). A machine learning outcome follows the same scheme as the features that make up the dataset. The value type scheme is shown in Figure 4-2.

Figure 4-2: Machine Learning Outcomes

Note The feature value types mentioned in the above diagram can be explained as:

Integer: Any non-decimal whole numbers e.g. 0, 1, 25, etc. Float: All non-rounded decimal numbers e.g. 0.1, 43.78, 93.156, etc. Boolean: True or False, otherwise 0 or 1. Nominal: Includes characters without any rank or order, e.g. p, a, k, etc. Ordinal: Includes numbers with a rank or order, e.g. 1st, 2nd 3rd, etc.

API’s

The TensorFlow.js library comprises of two separate API’s: the Layers API and the Core API. Depending on your background and what you want to achieve, an API is selected in TensorFlow.js. The available API’s in the TensorFlow.js library are shown in Figure 4-3.

Figure 4-3: TensorFlow.js API’s

Layers API

As illustrated in Figure 4-3, the Layers API can be used in one of the two following ways:

1. Sequential (tf.sequential(config?))

2. Functional (tf.model(args))

The Layers API is ideal in most situations especially if the developer has prior experience with machine learning experience using Keras. Regardless of its type, the Layers API enables creation of connected layers.

Sequential

The syntax for creating a linear stack of layers is tf.sequential(config?). Moreover, a sequential model is one where the outputs of a layer form the inputs of the next higher layer. The question mark (?) after the config parameter or argument signifies that it is optional.

Functional

Functional models are created using tf.model(args), are more generic than sequential models, and comprise an arbitrary graph of layers.

Core API

The Core API allows for creation of a machine learning model using low-level operations that enable developers the maximum flexibility and control.

Code

As of the writing of this book, the JavaScript API available for TensorFlow.js is version 2.0.1, captured in Figure 4-4.

Figure 4-4: TensorFlow.js API v2.0.1

Note The leaves in Figure 4-4 represent the categories of methods and not namespaces. These categories allow for grouping of objects and methods available in the TensorFlow.js library. You can view more details of the classes and methods by looking at the TensorFlow.js online help at https://js.tensorflow.org/api/latest/

The methods in each of the classes listed above can be analyzed by the help available online. The API summarized in the figure above is explained further below.

Tensors

Machine learning solutions require data for training purposes, and the data submitted to a machine learning library is in the form of a tensor with three properties; Rank, Shape, and Data Type. Figure 4-5a illustrates the types of tensors presently available in the TensorFlow.js API along with their properties.

Figure 4-5a: Tensor Types

The first category comprises of a scalar (tf.scalar) which is a simple value with 0 dimensions and a variable (tf.variable) with an initial and possibly final value, provided the shape of the initial and final values are the same. The following code snippets create a value of each type explained above.

Listing 4-1a: Using tf.scalar in code

// Define a scalar value
let val = tf.scalar(11.55);
document.getElementById('message').innerHTML = val;

Listing 4-1b: Using tf.variable in code

// Define a variable value
let val = tf.variable(tf.tensor([50, 100, 150]));
val.assign(tf.tensor([450, 500, 550]));
document.getElementById('message').innerHTML = 'Final Value: ' + val;

If the final value of tf.variable does not match the dimensions of the initial value, an error is generated, as shown in Listing 4-1c below.

Listing 4-1c: Generating an error when using tf.variable

// Define a variable value
try {
 // Define a variable value
 let val = tf.variable(tf.tensor([50, 100, 150, 200]));
 // the following statement generates an error
 val.assign(tf.tensor([450, 500, 550]));
 // this statement does not get executed
 document.getElementById('message').innerHTML = 'Final Value: ' + val;
}
catch(err) {
 // the statements in the catch block get executed when an error occurs
 document.getElementById('message').innerHTML = 'ERROR: ' + err.message;
}

The second category comprises methods that can be used to create and initialize values. You can refer either to code examples online or in the listings below.

Listing 4-1b: Using tf.clone in code

// Define a tensor
let val = tf.tensor([25, 29, 31, 33, 37]);
// Clone the tensor
let val2 = val.clone();
document.getElementById('message').innerHTML = val2;

Listing 4-2b: Using tf.fill in code

let val = tf.fill([5, 3], 100);
document.getElementById('message').innerHTML = val;

The code in Listing 4-2b creates a 5 x 3 tensor with all the values initialized to 100 using the tf.fill command.

Listing 4-2c: Using tf.complex in code

let realVal = tf.tensor1d([5.23, 7.51]);
let imagVal = tf.tensor1d([9.35, 1.86]);
let complexVal = tf.complex(realVal, imagVal);
document.getElementById('message').innerHTML = complexVal;

Listing 4-2d: Using tf.real in code

let realVal = tf.tensor1d([5.23, 7.51]);
let imagVal = tf.tensor1d([9.35, 1.86]);
let complexVal = tf.complex(realVal, imagVal);
document.getElementById('message').innerHTML = tf.real(complexVal);

Listing 4-2e: Using tf.imag in code

let realVal = tf.tensor1d([5.23, 7.51]);
let imagVal = tf.tensor1d([9.35, 1.86]);
let complexVal = tf.complex(realVal, imagVal);
document.getElementById('message').innerHTML = tf.imag(complexVal);

A complex number can be represented by the form a+bi where a is real and b is imaginary. The code snippets in listings 4-2c, 4-2d, and 4-2e can be seen in tandem since they all use the same tensor values. If you view the output of the code in Listing 4-2c, the imaginary part is represented with the letter j.

Listing 4-2f: Using tf.ones in code

let val = tf.ones([5, 3]);
document.getElementById('message').innerHTML = val;

The code in Listings 4-2b and 4-2f is similar in that both the snippets create a 5x3 tensor. However, the code in Listings 4-2f initializes all the values in the tensor to one.

Listing 4-2g: Using tf.onesLike in code

let val = tf.tensor([13, 25, 34, 49, 57]);
document.getElementById('message').innerHTML = val.onesLike();

Listing 4-2h: Using tf.range in code

let val = tf.range(95, 163, 3);
document.getElementById('message').innerHTML = val +
 "<br />Size of tensor: " + val.size;

The above code in Listing 4-2h, in addition to displaying the value, also displays the size of the tensor. Since the range may contain dozens or possibly hundreds of values, the output displays the range notation (…) to show the values in the tensor.

Listing 4-2i: Using tf.zeros in code

let val = tf.zeros([5, 3]);
document.getElementById('message').innerHTML = val;

Similar to the code in Listing 4-2f, the code above in Listing 4-2i also creates a 5x3 tensor but initializes all the values to zero.

Listing 4-2j: Using tf.zerosLike in code

let val = tf.tensor([13, 25, 34, 49, 57]);
document.getElementById('message').innerHTML = val.zerosLike();

The only difference between the code in Listing 4-2g and the one in Listing 4-2j is that in Listing 4-2j, all values of the newly created tensor are initialized to zero.

A tensor can refer to a collection of values and has a particular set of dimensions. The third category is shown in Figures 4-5b, 4-5c, 4-5d, and 4-5e, where a tensor is simply a container for a set of values passed to a machine learning model in code. A summarized view of various tensors explained in this chapter’s text is depicted in the following table.

Tensor Name	Rank	Shape	TensorFlow.js Example
Scalar	0	0	11.55
Rank 1 (Vector)	1		[1, 2, 3, 4, 5]
Rank 2 (Matrix)	2		[1, 2, 3, …, 23, 24, 25], [5, 5]
Rank 3 (Cube)	3		[1, 2, 3, …, 73, 74, 75], [3, 5, 5]
Rank 4	4		[1, 2, 3, …, 373, 374, 375], [5, 3, 5, 5]
Rank 5	5		[1, 2, 3, …, 1873, 1874, 1875], [5, 5, 3, 5, 5]
Rank 6	6		[1, 2, 3, …, 9373, 9374, 9375], [5, 5, 3, 5, 5, 5]

1, 2, and 3 are all arrays that store a set of numeric values; Rank 1 is a simple array whereas Rank 2 and Rank 3 are both arrays that are used to store images; Rank 2 for images that exist in gray-scale with values ranging from 0 to 255, and Rank 3 for color images that can be represented by RGB (red, green, and blue) values. Ranks 2 and 3 are also called as matrices.

Figure 4-5b: Rank 1, Rank 2, and Rank 3 Tensors

A Rank 4 tensor or matrix, shown in Figure 4-5c, is similar to a Rank 3 tensor and mainly used to store a one-dimensional array of colored images.

Figure 4-5c: Rank 4 Tensor

A Rank 5 tensor is also referred to as a cube of cubes and is shown in Figure 4-5d.

Figure 4-5d: Rank 5 Tensor

Lastly, a Rank 6 tensor is illustrated in Figure 4-5e.

Figure 4-5e: Rank 6 Tensor

Note You can choose to skip the rest of this section for now since it explains the categories that exist in the API, and come back to it after you have learnt the syntax used for programming with the library, explained in the next section.

Creation

Utility functions for creation of tensors to use for training machine learning solutions in the web browser using the TensorFlow.js library.

Classes

Methods of objects created by classes in the previous category.

Transformation

Methods of the tf class for applying tensor transformations in code.

Slicing and Joining

Methods of the tf class for slicing an existing tensor value, or for joining two tensor values to form a new tensor.

Random

Function to create a new tensor object from the specified distribution.

Models

Models represent connected layers that are used to predict outcomes after they (the models) have been trained with the available dataset.

Creation

Methods to create models using available API’s as illustrated in Figure 4-3.

Inputs

Factory to create an input for a model when tf.model is used.

Loading

Functions or methods to load a model or files using the web browser.

Management

Functions or methods to list, copy, move, and remove machine learning models.

Serialization

Method to register a model’s layers for serialization or deserialization.

Classes

Classes and Objects to create and use machine learning models.

Op Registry

Methods to register and de-register operations for a machine learning model.

Layers

A machine learning model comprises of multiple Layers, and each layer transforms its input to output by performing a computation on the input.

Advanced Activation

Advanced activation functions in the tf.layers namespace.

Basic

Basic activation functions in the tf.layers namespace.

Convolutional

Convolutional Neural Network (CNN) methods for convolutional layers in the tf.layers namespace.

Merge

Methods to perform operations on a model’s layers in the tf.layers namespace.

Normalization

Normalization methods for a model’s previous layer in the tf.layers namespace.

Pooling

Pooling operations in the tf.layers namespace for various types of data.

Recurrent

Factory methods for recurrent layers in the tf.layers namespace.

Wrapper

Wrapper methods for in the tf.layers namespace.

Classes

Classes and methods for layers in a machine learning model in the tf.layers namespace.

Inputs

Method for specifying an input layer for a tf.sequential model.

Padding

Method to add padding to image tensor.

Noise

Methods to add dropout to inputs for model training.

Mask

Method to skip timestep in sequential layers.

Operations

Represent tensor operations in the form of mathematical computations.

Arithmetic

Methods that represent arithmetic operations that act on tensors to produce new tensors.

Basic Math

Methods representing operations in basic math.

Matrices

Arithmetic operations that act on matrices to produce new tensors.

Convolution

Methods that represent convolution neural network operations for an input tensor.

Reduction

Methods that return a tensor after performing a logical or mathematical operation on an input tensor.

Normalization

Methods for normalization of an input tensor or activation function to return an output tensor.

Images

Methods to perform various operations on an input image tensor.

RNN

Methods to calculate the next state and output of one or more LSTM cells.

Logical

Methods that represent logical operations on a value, two input tensors, and/or a condition.

Scan

A method to calculate the cumulative sum of tensor.

Evaluation

Methods to calculate the confusion matrix or the largest entries along the last dimension.

Segment

A method to compute the sum along segments of a tensor.

Moving Average

A method to compute the moving average of a variable.

Slicing and Joining

Methods to slice and join and return the output in the form of a tensor.

Spectral

Methods in the tf.spectral namespace that compute different types of Fourier transforms on an input tensor.

Dropout

A method to calculate dropout from an input tensor.

Signal

Methods in the tf.signal namespace to perform operations on a signal.

Linear Algebra

Methods in the tf.linalg namespace to perform linear algebra operations on the input tensor.

Training

Represents functions to perform model training and calculate gradients, deep learning functions to initialize network parameters and then updating those parameters by minimizing a loss function.

Gradients

Methods to calculate gradients based on input functions defined using the fat arrow (=>) notation.

Optimizers

Methods that represent neural network optimizers in the tf.train namespace.

Losses

Methods to calculate loss between two tensors and return the output as a tensor.

Classes

Class that represents the return type of optimizer functions.

Performance

Represents methods that calculate telemetry and improve performance of classes and operations.

Memory

Methods to free up memory after code has executed.

Timing

Methods to return a JavaScript promise and calculate the execution time of some code.

Profile

Method to calculate the memory use of an anonymous function.

Environment

Methods that are global singleton functions to manipulate environment values.

Constraints

Methods in the tf.constraints namespace used to constrain the weights.

Classes

Base class for functions in this category.

Initializers

Methods in the tf.initializers namespace used to define the starting values of weights, biases, and kernels.

Classes

Base class for functions in this category.

Regularizers

Methods that can be attached to various components of a layer in a neural network model to keep weight values to a minimum during training.

Data

Represents methods to ingest data from a local or remote source in different formats, and prepare the ingested data for machine learning.

Creation

Methods to create data in various formats for machine learning, in the tf.data namespace.

Operations

Method to create a zip file from the input datasets.

Classes

Represent classes of supported data element types.

Visualization

Classes and methods to visualize data in the web browser.

Util

Utility methods in the tf.util namespace.

Browser

Method to convert a browser object to pixels and vice versa.

Backends

Represents methods to initialize new or manipulate existing backends, used for creating tensors and performing operation on those tensors.

Metrics

Methods that represent metric functions.

Callbacks

Method that represents a factory function to stop training.

TensorFlow.js Syntax

The TensorFlow.js syntax is also explained in its documentation online. The following text, however, is a concise explanation of the Layers API. Layers are central to a neural network model and a view of the neural network model in Figure 2-9b is illustrated below in Figure 4-6 below.

Figure 4-6: Neural Network Layers

Refer to Chapter 2 for a re-cap of the theoretical aspects of neural networks if need be; the text below is about the syntax and usage of the TensorFlow.js library. The following section has the following sub-sections:

1. Define neural network

   1. 1 Input layer

   2. 1 or more hidden layers

   3. 1 output layer

2. Compile the model

   1. Specify the optimizer

   2. Select the loss function

   3. Lay-down a list of metrics

3. Train the model

   1. Specify number of epochs

   2. Indicate a batch size

   3. Give the name of a call-back function to use

4. Make a prediction

Defining the Neural Network

Defining a neural network refers to creating a sequence of layers in network, along with the inputs from the previous layer and the activation function, shown in Figure 4-6 above and represented in JavaScript and TensorFlow.js code using the sequential API model as follows:

(For a complete list of parameters and their values checkout the online help at https://js.tensorflow.org/api/latest/#layers.dense)

Listing 4-4a: Defining a Sequential Neural Network Model in TensorFlow.js

// Define a sequential neural network model
const model = tf.sequential({
 layers: [
 // Define the first layer
 tf.layers.dense({inputShape: [4], units: 5, activation: 'relu'}),
 // Define the second layer
 tf.layers.dense({units: 3, activation: 'softmax'}),
 ]
});
// Display model summary
model.summary();

The code in Listing 4-4a can also be re-written as shown below in Listing 4-4b.

Listing 4-4b: Defining a Sequential Neural Network Model in TensorFlow.js

// Define a sequential neural network model
const model = tf.sequential();
// Define the first layer
model.add(tf.layers.dense({inputShape: [4], units: 5, activation: 'relu'}));
// Define the second layer
model.add(tf.layers.dense({units: 3, activation: 'softmax'}));
// Display model summary
model.summary();

The code in Listings 4-4a and 4-4b can be written using the functional model API as shown in Listing 4-5:

Listing 4-5: Defining a Functional Neural Network Model in TensorFlow.js

// Define a functional neural network model
const input = tf.input({shape: [4]});
// Define the first layer
const layer1 = tf.layers.dense({units: 5, activation: 'relu'}).apply(input);
// Define the second layer
const layer2 = tf.layers.dense({units: 3,
 activation: 'softmax'}).apply(layer1);
// Create the model
const model = tf.model({inputs: input, outputs: layer2});
// Display model summary
model.summary();

Note The text of this book uses the Layers API (Sequential and Functional) during the remaining chapters. Refer to online resources if you want to use the Core API in your code.

The last statement in listings 4-4a, 4-4b, and 4-5 i.e. model.summary() simply displays a summary of the created model in the web browser console tab (developer toolbar) as:

 _________________________________________________________________
 Layer (type) Output shape Param #
 =================================================================
 dense_Dense1 (Dense) [null,5] 25
 _________________________________________________________________
 dense_Dense2 (Dense) [null,3] 18
 =================================================================
 Total params: 43
 Trainable params: 43
 Non-trainable params: 0
 _________________________________________________________________

Compiling the Model

After the layers of a neural network are defined, the next step is to compile the model. Compiling a machine learning model means that the specified option values are validated.

Refer to the online help here: https://js.tensorflow.org/api/latest/#tf.LayersModel.compile

Before a model can be trained with data, it must first be compiled with the options as arguments to the compile function:

• Optimizer (https://js.tensorflow.org/api/latest/#Training-Optimizers)

• Loss (https://js.tensorflow.org/api/latest/#Training-Losses)

• Metrics (https://js.tensorflow.org/api/latest/#Metrics)

Optimizer

Refer to sub-section titled ‘Optimizer’ in Chapter 2, in addition to Figure 2-12b. You have the option to specify the optimizer in the compile function as either a string or as a call to a factory function that also requires a learning rate (also covered in Chapter 2). Refer to Figure 4-7a (an extension of Figure 2-12b) to see a list of optimizers in string format, as well as the factory function name in parenthesis that can be used in TensorFlow.js code.

Figure 4-7a: Name of the Optimizer as a String (in quotes) or Factory Function Name

Loss Function

To extend what was said in Chapter 2 in the sub-section ‘Loss/Cost Function’ under ‘Deep Learning Components’ and Figure 2-12c

Figure 4-7b: Fully-Qualified names of Loss Functions

Metrics

Metrics allow a machine learning model to be evaluated for accuracy. Keep in mind, however, that each metric applies to the type of machine learning problem being solved. A complete list of metrics used by the TensorFlow.js library are given below in Figure 4-7c.

Figure 4-7c: The Metrics in String (in quotes) or a the Loss/Metric Function Name

Note Code with the .compile statement is given in the following sub-section.

Train the Model

After you define the layers in a model, it is time to compile and train the model. Models can be trained using either the .fit or .fitDataset methods. The difference between the two is that .fit is used when the training data is composed of a large tensor, whereas .fitDataset is used for the Dataset object. Refer to the code snippets in Listings 4-6a and 4-6b.

Listing 4-6a: Compile and Train a Neural Network Model using the .fit method

function runCode() {
 // Define the model
 const model = tf.sequential({
 layers: [tf.layers.dense({units: 1, inputShape: [10]})]
 });
 // Compile the model
 model.compile({optimizer: 'sgd',
 loss: 'meanSquaredError',
 metrics: ['mse']});
 // Train the model
 model.fit(tf.ones([8, 10]), tf.ones([8, 1]), {
 batchSize: 1,
 epochs: 3
 });
 alert("Training completed.");
}

Listing 4-6b: Compile and Train a Neural Network Model using the .fitDataset method

function runCode() {
 const array1 = [
 [0.41804, 0.52203, 0.25374, 0.70718, 0.7418],
 [0.65186, 0.10075, 0.91624, 0.13423, 0.50037],
 [0.48045, 0.84411, 0.53332, 0.4084, 0.7637],
 [0.86752, 0.27962, 0.52211, 0.54758, 0.49345],
 [0.56164, 0.45481, 0.8307, 0.48488, 0.40333],
 [0.48509, 0.58096, 0.63123, 0.53886, 0.58651]
 ];
 const array2 = [98, 82, 94, 42, 22];
 // Convert arrays to datasets
 const dataset1 = tf.data.array(array1);
 const dataset2 = tf.data.array(array2);
 // Combine datasets, create sample batches, and shuffle the data
 const input = tf.data.zip({xs: dataset1,
 ys: dataset2}).batch(6).shuffle(5);
 // Define the model
 const model = tf.sequential({layers: [tf.layers.dense({units: 1,
 inputShape: [9]})]
 });
 // Compile the model
 model.compile({optimizer: 'sgd',
 loss: 'meanSquaredError'});
 // Train the model
 model.fitDataset(input, {
 epochs: 4
 });
 alert("Training completed.");
}

Make a Prediction

The last step after training the model is using that model to make a prediction. The code in Listing 4-6c is an extension of the code in Listing 4-6a.

Listing 4-6c: Making a prediction using the .predict method

function runCode() {
 // Define the model
 const model = tf.sequential({
 layers: [tf.layers.dense({units: 1, inputShape: [10]})]
 });
 // Compile the mode
 model.compile({optimizer: 'sgd',
 loss: 'meanSquaredError',
 metrics: ['mse']});
 // Train the model
 model.fit(tf.ones([8, 10]), tf.ones([8, 1]), {
 batchSize: 1,
 epochs: 3
 });
 // Make a prediction
 let result = model.predict(tf.ones([8, 10]), {batchSize: 4})
 alert("Prediction completed.");
 document.getElementById('message').innerHTML = result;
}

TensorFlow.js Ecosystem

I do not intend to dive deep into the ecosystem of the TensorFlow.js library. Keep in mind that other JavaScript libraries might get added to the ecosystem at some point in the future, but the diagram below will largely remain the same.

Figure 4-8: TensorFlow.js Ecosystem

Looking at Figure 4-8 bottom-up, once a reference to the TensorFlow.js library has been added to the web page code, developers can either train a model from scratch or use an existing model in the code to perform a prediction. Refer to the following section titled ‘Machine Learning with TensorFlow.js’ for an explanation of using either an existing machine learning model, or a model that has been trained using a custom neural network.

Machine Learning with TensorFlow.js

Refer to the section titled ‘TensorFlow.js Process’ in Chapter 5 to view a list off all activities in the machine learning workflow. The following sections cover creation of a machine learning models using each of the following methods in the TensorFlow.js library:

1. Train a model from scratch

2. Use pre-trained models

3. Transfer learning

Train a Model from Scratch

Refer to the section titled ‘Training a Model in a Web Browser’ in Chapter 5 to learn about creating machine learning solutions in the web browser using TensorFlow.js. Since this chapter deals mainly with the syntax that the TensorFlow.js library uses, pay close attention to the sequence of commands that are used to train a model. Refer to the code in Listing 4-6a to get an understanding of the training process in the TensorFlow.js library.

Inference

todo

Training

Todo

Note The next two sections deal with pre-trained models and transfer learning. While similar in theory, the two approaches differ in implementation in that the models in transfer learning are pre-trained but require additional training data to predict an outcome.

Use Pre-trained Models

Todo

Conversion

todo

Import

todo

Transfer Learning

Transfer learning refers to re-using a machine learning model (created for one problem) to solve a different relatable problem. It is used mainly for computer vision and natural language processing (NLP) tasks in deep learning because of the time and resources required to train neural networks from scratch. Based on the data in the following table, the visual illustration in Figure 4-9 has been created, and shows some pre-trained models that can be used for transfer learning.

Datatypes	Model	Description
Text	Text Toxicity	Text sentiment analysis of a comment or tweet
	Universal Sentence Encoder	Segmentation or classification of text
Sound	Speech Commands	Classification of speech commands
Numeric	KNN Classifier	Number classification using k nearest neighbor
Images or Video	MobileNet	Image classification using textual labels
PoseNet	Human pose estimation
	BodyPix	Segmentation of person or body parts
	Coco SSD	Detection of multiple objects in an image
	DeepLab	Segmentation of various items in an image

Note The meanings of the terms ‘Segmentation’ and ‘Classification’ are different. Segmentation refers to separating out similar items or objects from the given data, whereas classification (or detection) goes a step further and places each object in a class or category. A good example of classification is shown in Listing 4-6.

Figure 4-9: Pre-trained Models for Transfer Learning

Note The diagram in Figure 4-9 makes use of the datatypes shown in Figure 2-6b.

Refer to the GitHub Repo at https://github.com/tensorflow/tfjs-models for a list of models, their source, and a live preview.

Tip For various artifacts pertaining to machine learning like pipelines and sample training data, visit the Google AI Hub at https://aihub.cloud.google.com For machine learning models created using Caffe, check out the wiki page on GitHub at https://github.com/BVLC/caffe/wiki/Model-Zoo

The above diagram in Figure 4-9 classifies the pre-trained models as per the types of data used to train them. Since model training with neural networks requires a lot of resources, especially one that requires processing of images, it makes a lot of sense to skip the training part and go directly to re-using the models trained on large datasets using state-of-the-art hardware by third parties.

Refer to the code in Listings 4-7a and 4-7b for a problem involving transfer learning and a pre-trained model known as ‘mobilenet’. Listing 4-7a uses the model as-is without any training data to classify an image.

Listing 4-7a: Image Classification using a Pre-trained Model

let model;
async function predictFromImage() {
 document.getElementById('message')
 .innerText = "Classification in progress. Please wait...";
 // Load mobilenet Model
 model = await mobilenet.load();
 // Make a prediction on the Image
 const imgElement = document.getElementById("ImageElement");
 const result = await model.classify(imgElement);
 document.getElementById('message').innerText = result[0].className;
}

Listing 4-7b: TODO

// TODO

Process, Components and Backends

Before I end this chapter, I wanted to leave you with the following diagrams which illustrate the TensorFlow.js library visually.

Figure 4-10a shows a high-level view of machine learning process, whereas Figure 4-10b, although similar to the previous diagram, displays the components that make up a neural network model. Figure 4-10c illustrates the code components of TensorFlow.js while the other areas have been grayed-out. Pre-trained models and transfer learning are shown in Figure 4-10d. Lastly, Figures 4-10e and 4-10f illustrate TensorFlow.js components and backends respectively.

Figure 4-10a: Machine Learning Process

Figure 4-10b: Neural Network Parameters

Figure 4-10c: Process of Training a Model using TensorFlow.js

Figure 4-10d: Pre-trained Models and Transfer Learning

Figure 4-10e: TensorFlow.js Components

Figure 4-10f: TensorFlow.js Backends

Key Takeaways

The chapter dives into the underlying topics of deep learning, neural networks, and the TensorFlow.js library, and covers the following:

• Definition and concepts of deep learning

• TensorFlow.js functions used for creating neural networks and learning from data

• JavaScript code for creating neural networks