Train a Neural Network from Scratch with TensorFlow.js

Chapter 5

Training a machine learning model involves testing and validating that model with the available data, but not the data used to create the model.

An end-to-end machine learning workflow is illustrated in Figure 2-6. This chapter focuses on step 3 of the workflow i.e. train. Before we begin training a neural network from scratch with TensorFlow.js, make sure you understand what a deep neural network is, discussed in Chapter 2 in the section titled ‘Deep Learning Components’.

Machine Learning Workflow

As illustrated in Figure 2-6a, the machine learning workflow is divided into the following four steps or stages; Ingest, Prepare, Validate, and Publish. Depending on whether it makes sense to update the machine learning model with new or updated data, the workflow or life-cycle will move from the last step i.e. Publish, back to the first step or stage.

Ingest

In this step, data is collected or consumed from different sources (see Figure 2-3). It makes sense to copy data to a central location in the machine learning library’s line of sight; or to a location on the network where the ML library has access to the data.

Prepare

Data preparation involves taking data collected from various sources, performing an exploratory data analysis (EDA) on those facts and figures, and generating a dataset for the next step in the workflow.

Validate

The name of this stage is a bit misleading since it requires much more than just validating the gathered and prepared data from the previous two steps, and involves training and generating a machine learning model. For TensorFlow.js, this has been covered later on in this chapter.

Publish

To allow different applications and processes to make use of the machine learning model, it must first be published. Models are also tracked after they are published. Tracking enables a model or its underlying data to be versioned and/or updated if need be. Model publishing is covered in Chapter 11.

TensorFlow.js Process

TensorFlow.js can be setup and executed in a web browser using the five steps below. Figure 5-1 shows this five-step machine learning process followed by TensorFlow.js, from setting up the script to testing and evaluating the model.

The reason steps 3 and 6 (Define Architecture and Deploy Solution) are grayed-out is because they lie outside the TensorFlow.js library, even though they are part of the process.

Each on the steps in the TensorFlow.js process is described below.

Setup Script

Every HTML file that needs to access the machine learning code must have the reference to the TensorFlow.js script files. Listing 3-1 mentions reference scripts for both, the TensorFlow.js script file, and the TensorFlow.js visualization script file.

Get Data

This step comprises mainly of 2 sub-steps: data acquisition and data visualization. Data acquisition involves getting the data and cleaning it if necessary, whereas visualization refers to creating a visual representation of the loaded data that is easy to understand.

Figure 5-1: TensorFlow.js Process

Note Figure 5-1 borrows steps from the Machine Learning Workflow illustrated in Figure 2-6a. While the names of a majority of the steps are the same, machine learning is typically performed on the server, and it involves the steps carried out by different people. Conversely, all the steps in a web browser-based solution are performed by the same person. Also, the TensorFlow.js script file does include any steps or code for deployment of the machine learning solution; therefore, the ‘Deploy Solution’ box in the above diagram has been grayed-out.

Define Architecture

Software architecture determines the components of a software application and their layout to solve the problem at hand. The architecture of a machine or deep learning solution defines which machine learning algorithms will be trained to find a solution to an analytics problem. The architecture is defined after data is loaded and cleansed for training.

Tune Parameters

Two sets of parameters

Tuning Parameters (also known as hyperparameter optimization)

Train Model

Training is what this chapter is all about. Training a model with the loaded data and algorithms entails teaching a model to predict, forecast, or infer the outcome(s) from the training data. Furthermore, training must be performed on a new set of data from scratch, since it does not suffice to train a model only once on the given data and expect the model to perform equally well on another dataset.

Test and Evaluate

Once a model has been trained with training data, it needs to be tested and evaluated for accuracy with some testing data, which is separate from the training data. The algorithm with the most accurate outcome with the test data is chosen as the resulting model. This step is the focus of this chapter.

Deploy Solution

Deployment pertains to moving the solution from lower environments to the production servers where they are used by the end-users to perform machine learning prediction using the TensorFlow.js library, using actual data.

Note This chapter covers training the model with the available training data. The rest of the steps in the TensorFlow.js process have been covered in Chapter 6. Also, the TensorFlow.js script does not include anything for installed deployment in the production environment, beyond copying the script and markup files. Hence, nothing pertaining to deployment of the solutions are covered in this chapter or the next. Deployment is, however, covered in Chapter 11.

Training a Model in a Web Browser

Once data has been loaded in a web browser, it is used to train and validate a model. Refer to Figure 5-2 below for a list of end-results for each step.

Figure 5-2: TensorFlow.js Activity End-Results

As shown above, training in TensorFlow.js results in a machine learning model which exists as a JSON (JavaScript Object Notation) file. In order to train a model, two artifacts are required; the training dataset and a neural network, shown in Figure 5-3.

Note The training dataset is a subset of the entire dataset and is explained in more detail in the section titled ‘Model Training’ in Chapter 6 and also shown visually in Figure 6-4. For more information on neural networks, refer to Chapter 2.

Figure 5-3: Artifacts involved in the Train Step of TensorFlow.js

Visualizing Training

[todo]

Key Takeaways

The chapter dives into the underlying topics of deep learning, neural networks, and the TensorFlow.js library, and covers the following:

• Definition and concepts of deep learning

• TensorFlow.js functions used for creating neural networks and learning from data

• The parameter values for creating neural networks