I’ve worked as a web developer for a while, and most times, I’m amazed at how web apps such as Facebook and Instagram are able to detect and recognize objects in images. I reckon a machine learning model like MobileNet makes this possible, but how do you deploy this model to the web? Deploying machine learning (ML) models to the web involves the process of taking an ML model that has been trained and tested offline and making it available for use in a web application. This process typically involves several steps, including selecting an appropriate web framework, setting up the necessary infrastructure, and integrating the model with the web application. Once the model is deployed, it can be used to make predictions or provide other intelligent services to web app users.
Importance of Integrating ML Models into Web Applications
Integrating ML models into web applications can provide a number of benefits, including:
- Improved User Experience: ML models can be used to personalize the user experience, making it more relevant to the individual user’s needs and preferences.
- Enhanced Functionality: ML models can add new functionality to web applications, such as natural language processing or image recognition.
- Increased Efficiency: ML models can automate certain tasks, reducing the manual work required to complete them.
- Enhanced Decision-Making: Machine learning models can offer valuable guidance and suggestions, enabling users to make more informed choices.
In this article, I’ll help you get familiar with machine learning models and popular frameworks like TensorFlow. We’ll explore some of these models, paying special attention to TensorFlow.js. After that, we’ll integrate a MobileNet model into a web app. So, let’s jump right in!
Types of ML models
An ML model is like a computer program trained to learn from examples. It’s like a student who is taught using many examples of a problem until they can solve similar problems on their own. The more examples the model is given, the better it becomes at making predictions or decisions based on new data. For example, an ML model might be trained to recognize different types of animals in pictures. After being shown many pictures of different animals and told what they are, the model can identify animals in new pictures that it hasn’t seen before.
This section will provide an overview of each type and offer illustrative examples for better understanding.
1. Supervised Learning: Supervised learning is the most common form of machine learning, where the model is trained on a labeled dataset. This means that the input data is paired with the correct output. The model learns to map input data to the desired output through a series of iterations, adjusting its parameters to minimize the error between its predictions and the actual outputs.
Example: Suppose we want to create a model that can recognize handwritten digits. We would train this model using a dataset containing images of handwritten digits (input) along with their correct numerical labels (output). Once trained, the model should be able to identify the correct digit when presented with a new, unlabeled image.
Subtypes of Supervised Learning
- Classification: The model is trained to categorize input data into two or more predefined classes. Examples: email spam detection, image recognition, and disease diagnosis.
- Regression: The model is trained to predict a continuous value instead of a discrete class. Examples: house price prediction, stock price forecasting, and demand forecasting.
Learn more about supervised learning:
Supervised Learning – Towards Data Science
Supervised Learning with scikit-learn
Supervised Learning algorithms
2. Unsupervised Learning
Unsupervised learning involves training a model on an unlabeled dataset, meaning the input data does not come with the correct output. The model must discover patterns or structures within the data without guidance, typically by finding similarities or differences among the data points.
Example: Suppose we have a dataset containing customer data for an e-commerce website. An unsupervised learning model can be used to group customers with similar behaviors, preferences, or demographics. This could be used to create targeted marketing campaigns or personalized product recommendations.
Subtypes of Unsupervised Learning
- Clustering: The model is trained to group data points based on similarity or proximity. Examples: customer segmentation, document clustering, and image segmentation.
- Dimensionality Reduction: The model is trained to reduce the number of features in the dataset while preserving the essential structure or relationships. Examples: principal component analysis (PCA), t-distributed stochastic neighbor embedding (t-SNE), and autoencoders.
Learn more about unsupervised learning:
Unsupervised learning demystified
Supervised vs Unsupervised Learning
3. Reinforcement Learning
Reinforcement learning is a type of machine learning where an agent learns to make decisions by interacting with its environment. The agent receives feedback in the form of rewards or penalties, which it uses to adjust its actions to maximize its cumulative reward over time. This process is often used to train models for tasks that require decision-making or control.
Example: Consider a self-driving car that needs to learn how to navigate traffic safely and efficiently. A reinforcement learning model can be employed to train the car by allowing it to interact with a simulated environment. The car receives positive rewards for reaching its destination and negative rewards for collisions or violating traffic rules. Over time, the model learns to make better decisions that maximize its cumulative reward.
Subtypes of Reinforcement Learning
- Model-Based Reinforcement Learning: The agent builds a model of the environment and uses it to plan its actions.
- Model-Free Reinforcement Learning: The agent does not have access to, or does not use, a model of the environment to make decisions. Instead, the agent learns an optimal policy or value function directly from its interactions with the environment.
Learn more about reinforcement learning:
A beginners guide to reinforcement learning
An Introduction to Reinforcement learning
Popular Machine Learning Frameworks
Several popular machine-learning frameworks make it easier for developers and researchers to build, train, and deploy machine-learning models. TensorFlow, PyTorch, and scikit-learn are three of the most widely used frameworks. This section will discuss each of these frameworks in detail, along with examples and use cases.
1. TensorFlow
TensorFlow is an open-source machine learning framework developed by Google Brain. It is designed to be flexible, efficient, and scalable, allowing users to build and deploy machine learning models on various platforms, including web, desktop, mobile, and cloud environments. TensorFlow supports deep learning and other machine learning techniques and is particularly popular in computer vision and natural language processing.
Use-Cases:
- Image Classification: TensorFlow can be used to build convolutional neural networks (CNNs) to classify images, such as identifying objects or animals in photos.
- Text Generation: TensorFlow can be employed to create recurrent neural networks (RNNs) or Transformer models for generating human-like text based on a given input or prompt.
- Recommender Systems: TensorFlow can be utilized to create models that provide personalized recommendations, such as movie or product recommendations, based on user preferences and behavior.
2. PyTorch
PyTorch is an open-source machine learning framework developed by Facebook’s AI Research lab (FAIR). It is based on the Torch library and is designed to provide flexibility, ease of use, and high performance for deep learning applications. PyTorch is particularly popular among researchers due to its dynamic computation graph and strong support for GPU acceleration, which makes it ideal for rapid prototyping and experimentation.
Use-Cases:
- Natural Language Processing: PyTorch can be used to create models for tasks like sentiment analysis, machine translation, and question-answering using RNNs, LSTMs, or Transformer architectures.
- Generative Adversarial Networks (GANs): PyTorch is well-suited for building GANs, which can generate realistic images, sounds, or other data by pitting a generator network against a discriminator network in a competitive setup.
- Reinforcement Learning: PyTorch can be employed to develop reinforcement learning models for tasks like game playing, robotic control, and autonomous navigation.
3. Scikit-learn
Scikit-learn is an open-source machine learning library built on top of NumPy, SciPy, and Matplotlib. It is specifically designed for traditional machine learning techniques rather than deep learning, and offers a simple, consistent API that makes it easy for users to experiment with various models and techniques.
Use-Cases:
- Supervised Learning: Scikit-learn provides a variety of algorithms for classification (e.g., logistic regression, support vector machines) and regression (e.g., linear regression, decision trees) tasks.
- Unsupervised Learning: Scikit-learn includes methods for clustering (e.g., k-means, DBSCAN), dimensionality reduction (e.g., PCA, t-SNE), and anomaly detection (e.g., isolation forest, one-class SVM).
Having covered the fundamentals of machine learning models, their types, popular frameworks, and respective use-cases, it’s time to delve into the practical aspects of integrating a machine learning model into a web application. We will be focusing on utilizing the TensorFlow machine learning framework, which is highly recommended for this purpose, owing to its versatility and compatibility with web environments.
TensorFlow provides a powerful JavaScript library called TensorFlow.js, specifically designed to facilitate developing and deploying machine learning models within web browsers and Node.js environments. This enables users to harness the capabilities of TensorFlow without leaving the comfort of their favorite web technologies.
Integrating the ML Model into the Web Application using TensorFlow.js
To start implementing a machine learning model using TensorFlow.js, you will first need to include the library in your project, either by adding it as a dependency via npm or by including a script tag in your HTML file. Once this is done, you can choose from a variety of pre-trained models, modify them as needed, or even create your own custom model using TensorFlow.js APIs. Finally, you can integrate the model into your web application, providing users with powerful machine learning capabilities right within their browser.
Tensorflow.js provides us with a set of pre-trained models. For the purpose of this article, we will work with some of these models.
TensorFlow.js MobileNet Model
The MobileNet model is a convolutional neural network (CNN) designed for efficient image classification tasks, particularly on mobile and embedded devices.
The primary purpose of the MobileNet model is to provide an efficient solution for image classification tasks while maintaining high accuracy. It can recognize and categorize objects within images, such as identifying animals, plants, vehicles, and other objects. MobileNet can be applied in various applications, including:
- Image recognition and tagging for photo organization
- Augmented reality (AR) apps that recognize and label real-world objects
- Object detection and tracking in video streams
- Real-time image classification on mobile devices
Pre-requisites
When you want to integrate a TensorFlow.js model into a web application, it’s a good idea to follow these steps:
- Load the model: Start by importing the TensorFlow.js model into your web application. This step typically involves fetching the model from a server or loading it from a pre-trained file.
- Preprocess the input data: Before feeding data into the model, it’s important to preprocess it according to the model’s requirements. This may involve resizing images, normalizing data, or converting data into a format the model can understand.
- Implement the prediction or inference functionality: Once the data is preprocessed, you can use the model to make predictions or inferences based on the input data. This process typically involves calling a specific function or method provided by the TensorFlow.js library to apply the model to your data.
- Post-process the model output: After obtaining the model’s predictions or inferences, you may need to process the results to make them more understandable or useful. This could involve converting the model’s output into human-readable labels, extracting the most relevant results, or further processing the data to meet your needs.’
Let’s get started.
1. Loading the model using TensorFlow.js
First, include the TensorFlow.js library in your HTML file by adding the following script tag:
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs"></script>
Next, load the MobileNet model using the load() method from the @tensorflow-models/mobilenet package. You can include this package in your project using npm or by adding a script tag:
<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/mobilenet"></script>
Now you can load the model in your JavaScript code:
const loadModel = async () => {
const model = await mobilenet.load();
console.log('Model loaded successfully');
return model;
};
This asynchronous function is responsible for loading the MobileNet model using the mobilenet.load() method from the TensorFlow.js MobileNet package. When the model is successfully loaded, it logs a message to the console and returns the model.
2. Preprocessing input data for the model
MobileNet expects input images to be 224×224 pixels. You can obtain an image from a file input, a canvas element, or an image element. In this example, we will use an image element with the ID inputImage. You will need to preprocess the image before feeding it to the model:
const preprocessImage = (imageElement) => {
const imageTensor = tf.browser.fromPixels(imageElement);
const resizedImageTensor = tf.image.resizeBilinear(imageTensor, [224, 224]);
return resizedImageTensor;
};
This function takes an image element as input and preprocesses it to meet the MobileNet model’s requirements. It first converts the image element to a TensorFlow.js tensor using tf.browser.fromPixels(imageElement). Then, it resizes the tensor to the expected dimensions of 224×224 pixels using tf.image.resizeBilinear(imageTensor, [224, 224]).
3. Implementing prediction/inference functionality:
Create a function that takes the preprocessed image tensor and feeds it to the model to obtain predictions:
const classifyImage = async (model, preprocessedImage) => {
const predictions = await model.classify(preprocessedImage);
return predictions;
};
This asynchronous function takes the loaded model and preprocessed image tensor as inputs. It then calls the model.classify() method to obtain predictions for the given image. The function returns these predictions.
4. Post-processing model output
The classify() function returns an array of predictions containing a className and probability. You can display the predictions in a user-friendly format, such as an HTML table or list. Here’s we will display the top 3 predictions in a list:
const displayPredictions = (predictions) => {
const predictionList = document.getElementById('predictions');
predictionList.innerHTML = '';
predictions.slice(0, 3).forEach((prediction) => {
const listItem = document.createElement('li');
listItem.textContent = `${prediction.className}: ${Math.round(prediction.probability * 100)}%`;
predictionList.appendChild(listItem);
});
};
This function takes the predictions array as input and displays the top 3 predictions in a user-friendly format. It first clears the predictions list, then iterates through the top 3 predictions and creates a list item for each. The list item contains the class name and the probability rounded to a percentage value. Finally, the list items are appended to the predictions list on the page.
Putting it all together
Now you can load the model, preprocess the input image, classify it, and display the predictions when an image is selected:
document.getElementById('inputImage').addEventListener('change', async (event) => {
const model = await loadModel();
const preprocessedImage = preprocessImage(inputImage);
const predictions = await classifyImage(model, preprocessedImage);
displayPredictions(predictions);
});
That’s it! The application can now preprocess user-uploaded images, classify them using the MobileNet model, and display the top 3 predictions to the user.
The complete code is on GitHub.
Output
Conclusion
In conclusion, deploying an ML model to the web requires a multi-step process, including selecting an appropriate web framework, setting up the necessary infrastructure, and integrating the model with the web application. However, the benefits of integrating ML models into web applications can be significant, including improved user experience, enhanced functionality, increased efficiency, and improved decision-making.
In this article, we have explored the process of deploying machine learning models to the web using popular frameworks like Tensorflow. We have demonstrated the steps in integrating a model into a web application using Tensorflow.js.
Moving on…
I encourage further exploration and experimentation with machine learning models and web frameworks to discover new ways to improve user experience, functionality, efficiency, and decision-making in web applications. With the right tools and techniques, you can create powerful and innovative web applications that take advantage of the latest advances in machine learning technology. So, keep exploring and experimenting to discover new possibilities in this exciting and rapidly evolving field.
Resources
TensorFlow.js Tutorials: https://www.tensorflow.org/js/tutorials
“Build and Deploy a Machine Learning Model with TensorFlow.js” by TensorFlow: https://codelabs.developers.google.com/codelabs/tensorflowjs-teachablemachine-codelab/index.html
“Machine Learning in the Browser with TensorFlow.js” by Gant Laborde: https://www.apress.com/gp/book/9781484252022
TensorFlow.js YouTube playlist by TensorFlow: https://www.youtube.com/playlist?list=PLQY2H8rRoyvwLbzbnKJ59NkZvQAW9wLbx
TensorFlow.js Pre-trained Models: https://github.com/tensorflow/tfjs-models
