Train a Deep Learning model for custom object image classification using Teachable Machine, convert it to a TFLite model, and finally deploy it on mobile devices using the sample TFLite image classification app from TensorFlow’s GitHub.
IMPORTANT
THIS IS FOR THE LATEST VERSION OF THE TENSORFLOW APP ON GITHUB. YOU CAN SWITCH TO PREVIOUS VERSIONS IN GITHUB. USE THE FOLLOWING VERSION FOR THE IMAGE CLASSIFICATION REFERENCE APP USED IN THIS TUTORIAL:
https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification
Roadmap
- ⁍ Collect the dataset of images.
- ⁍ Open Teachable-Machine’s image project
- ⁍ Train the model.
- ⁍ Evaluate the model.
- ⁍ Export the model
- ⁍ Create TFLite model for Android
- ⁍ Download the sample image classification app from TensorFlow and adjust it for your custom TFLite model.
Image classification or image recognition is a concept in which you showcase an image to the camera sensor of the device and it will tell you what is present in that image or tell us which class it belongs to.
In this tutorial, I will walk you through the custom image classification by training a simple deep learning model with the help of an exciting online tool by google: teachablemachine with google, and then exporting the model to the TensorFlow lite version which is compatible with an Android device. Then, finally, we will deploy this model onto an Android device.
Model
The training platform used for training our custom image classifier is the teachablemachine with google. This is an exciting platform for learning the deep learning training process with just a very few clicks. First, by uploading the different classes of objects from your system or using a webcam, then training it. Finally, after training, you can export the model of your choice. You can choose whatever format you want and download the model. I am training a model for mask image classification. This will be done in the 5 steps mentioned in the section below. The first 3 steps are the same as in the Train image classification model using Teachable Machine tutorial.
- Upload your dataset
- Train the model
- Preview the model / test the model using javascript
- Export the model in TFLite format
- Download the TFLite model and add metadata to it
- Adjust the TensorFlow sample Image classification for your TFLite models and run it
Requirements
- Your custom object image dataset with different classes.
- Android Studio 3.2 + (installed on a Linux, Mac, or Windows machine)
- Android device in developer mode with USB debugging enabled
- A USB cable (to connect the Android device to your computer)
LET’S BEGIN !!
( But first ✅Subscribe to my YouTube channel ???????? https://bit.ly/3Ap3sdi ????????)
Objective: Build a custom object Image classification android app using Teachable Machine.
To start, go to the Teachable-Machine site.
NOTE: TeachableMachine only has support for MobileNet model as of now.
Click on Get Started on the homepage and choose Image Project. You can also sign in using your google drive to save your model files otherwise you will lose all your images and model files if you close the tab or refresh the page.
Step 1. Upload your dataset
Create and upload the dataset to the teachablemachine and define the names of the classes accordingly. I am training a model for mask image classification with 2 classes viz., “with_mask” & “without_mask”.
Step 2) Train the image classification model
During training, you can tweak the hyperparameters like:
- No of epochs
- Batch size
- Learning rate
Click on Train Model after setting the values of the hyperparameters. This process will take a while depending on the number of images and epochs.
Important: During training, do not switch tabs as it will stop the training process.
Step 3) Preview your trained model or test the model using javascript
Once the training is over, you can preview your trained model in the preview window. You can either upload images to test or you can test using a webcam. See “preview” test results for both below.
USING WEBCAM
USING IMAGES
You can also test the model using javascript.
You can use your model for browser-based projects using the javascript format.
First, click on Export model. Next, go to the Tensorflow.js tab and click on Upload my model. When you upload your model, Teachable Machine hosts it at the given shareable link for free. You can share that link with anyone if you’d like them to use your model — anyone who has that link can use your model in their projects. Note that your model is published to Google servers, but the examples you used to make the model are not. Just your model — the mathematical program that predicts which class you’re showing it.
You will see the Javascript code snippet with your model’s link written in it. You can use this code to test your model on a browser.
You can also test your model online using the p5.js web editor tool. The web editor for p5.js is a JavaScript library with the goal of making coding accessible to artists, designers, educators, and beginners. To use it for your custom-trained model, select the p5.js code snippet tab and copy the code. Next, click on the p5.js Web Editor link shown in the pic below.
Once you open the p5.js web editor tool you will find the default code there for daytime and nighttime classification. Remove the entire old code and paste the new code you copied from your custom model’s p5.js code snippet and press play. Now, you can test your model using the p5.js web editor tool.
Step 4) Export the model in TFLite format
Finally, export the model in the form of TensorFlow lite format to deploy on mobile devices.
Click on Export Model and select the TensorFlow Lite tab. Next, choose the model format you want. You can download the quantized as well as the floating-point file format of TFLite.
NOTE: Teachable Machine only supports MobileNet model architecture as of now.
The .tflite file should be placed in the asset folder of the android project directory and change the name in the java file which is reading it accordingly(See Step 5)
Step 5) Download the TFLite and add metadata to it
The current latest version does not have a labels.txt file as the classes are added along with the metadata for all the models of the sample image classification app. We need to add metadata to our TFLite model to attach the classes and other necessary information.
You can use this colab notebook below to create metadata:
https://colab.research.google.com/drive/15bXIFWYGlujeSF_UBYkLqP6ia9TpJx23?usp=sharing
Also, check out the following 2 links to learn more about metadata for image classifiers.
- https://www.tensorflow.org/lite/convert/metadatahttps://www.tensorflow.org/lite/convert/metadata_writer_tutorial#image_classifiers
Step 6) Adjust the TensorFlow sample Image classification for your TFLite models and run it
Open the android_java version of the image classification app in Android Studio and adjust it for your custom model
(Build the app using Android Studio as described in the link below: https://github.com/tensorflow/examples/blob/master/lite/examples/image_classification/android/README.md)
⦿ Download the TensorFlow Lite examples archive from here and unzip it. You will find an image classification app inside
NOTE: I am using the android_java version of the image classification app in Android Studio. You can use the android kotlin version too. The changes there are similar to the ones shown below for the java version.
examples-master\lite\examples\image_classification\android_java
⦿ Next, build the project first to create the assets folder & download all the default models for the image classification sample app inside it.
Select Build -> Make Project and check that the project builds successfully. You will need Android SDK configured in the settings. The build.gradle file will prompt you to download any missing libraries.
NOTE: You can also create the assets folder manually. Since I will also be using the default models in the app I am building the project. If you don’t want to use the default downloaded models and just want your own model in the app, comment out the apply from:’download_models.gradle line in the Android app Module “build.gradle” file and create the assets folder manually inside the main directory location given below.
⦿ Next, copy your custom TFLite models with metadata inside the assets folder.
image_classification\android_java\app\src\main\assets
⦿ Next, make changes in the code as mentioned below.
➢ First, set the names of your TFLite model as modelName for each model version you have. You can set different models here but since we are using only floating-point & quantized models for MobileNet, I will add 2 more to the model names.
Add the names of your models to the modelName function in the ImageClassifierHelper.java file.
String modelName;
switch (currentModel) {
case MY_CUSTOM_MODEL:
modelName = "model.tflite";
break;
case MY_CUSTOM_MODEL_QUANTIZED:
modelName = "model_quant.tflite";
break;
case MODEL_MOBILENETV1:
modelName = "mobilenetv1.tflite";
break;
case MODEL_EFFICIENTNETV0:
modelName = "efficientnet-lite0.tflite";
break;
case MODEL_EFFICIENTNETV1:
modelName = "efficientnet-lite1.tflite";
break;
case MODEL_EFFICIENTNETV2:
modelName = "efficientnet-lite2.tflite";
break;
default:
modelName = "mobilenetv1.tflite";
}
Also, add it to the ImageClassifierHelper class as shown below. Set the order id number in the same order as the above step. Since I put my custom models at the first 2 lines which is array id 0 & 1, I have set them here to 0 & 1 and changed others accordingly.
public class ImageClassifierHelper {
private static final String TAG = "ImageClassifierHelper";
private static final int DELEGATE_CPU = 0;
private static final int DELEGATE_GPU = 1;
private static final int DELEGATE_NNAPI = 2;
private static final int MY_CUSTOM_MODEL = 0;
private static final int MY_CUSTOM_MODEL_QUANTIZED = 1;
private static final int MODEL_MOBILENETV1 = 2;
private static final int MODEL_EFFICIENTNETV0 = 3;
private static final int MODEL_EFFICIENTNETV1 = 4;
private static final int MODEL_EFFICIENTNETV2 = 5;
For Kotlin app version edit the modelName function and the companion object inside the ImageClassifierHelper.kt file.
val modelName =
when (currentModel) {
MY_CUSTOM_MODEL -> "model.tflite"
MY_CUSTOM_MODEL_QUANTIZED -> "model_quant.tflite"
MODEL_MOBILENETV1 -> "mobilenetv1.tflite"
MODEL_EFFICIENTDETV0 -> "efficientdet-lite0.tflite"
MODEL_EFFICIENTDETV1 -> "efficientdet-lite1.tflite"
MODEL_EFFICIENTDETV2 -> "efficientdet-lite2.tflite"
else -> "mobilenetv1.tflite"
}
companion object {
const val DELEGATE_CPU = 0
const val DELEGATE_GPU = 1
const val DELEGATE_NNAPI = 2
const val MY_CUSTOM_MODEL = 0
const val MY_CUSTOM_MODEL_QUANTIZED = 1
const val MODEL_MOBILENETV1 = 2
const val MODEL_EFFICIENTDETV0 = 3
const val MODEL_EFFICIENTDETV1 = 4
const val MODEL_EFFICIENTDETV2 = 5
}
➢ Second, Edit your strings.xml file to add your model names to the string array for model names in order to display it on the app page models dropdown list.
<string-array name="models_spinner_titles">
<item>My Custom Model</item>
<item>My Custom Model Quantized</item>
<item>MobileNet V1</item>
<item>EfficientDet Lite0</item>
<item>EfficientDet Lite1</item>
<item>EfficientDet Lite2</item>
</string-array>
The strings.xml file is inside the res\values directory.
..\image_classification\android_java\app\src\main\res\values
- ➢ Finally, connect an Android device to the computer and be sure to approve any ADB permission prompts that appear on your phone. Select
Run -> Run app.Test your app before making any other new changes or adding more features to it. Now that you’ve made a basic Image classification app using Teachable Machine, you can try and make changes to your TFLite model or add any customizations to the app. Have fun!
Read these TensorFlow image classification app GitHub Readme and Explore_the_code pages to learn more->
TensorFlow Lite image classification Android example application/Readme
TensorFlow Lite Android image classification example/Explore_the_code
My GitHub
Image Classification sample app
Mask dataset
Check out my Youtube Video on this!
CREDITS
Documentation / References
- ▸ Tensorflow Introduction
- ▸ Tensorflow Models Git Repository
- ▸ TensorFlow 2 Classification Model Zoo
- ▸ TensorFlow Lite sample applications
- ▸ TensorFlow Lite image classification Android example application
- ▸ Tensorflow tutorials
- ▸ Integrate image classifiers
- ▸ Adding metadata to TensorFlow Lite models
- ▸ TensorFlow Lite Metadata Writer API
- ▸ Create metadata command
- ▸ TensorFlow Lite image classification Android example application/Readme
- ▸ TensorFlow Lite Android image classification example/Explore_the_code
English