Machine learning has become part of day-to-day life. Small tasks like searching songs on YouTube and suggestions on Amazon are using ML in the background. This is a well-developed field of technology with immense possibilities. But how we can use it?

This blog is aimed at explaining how easy it is to use machine learning models (which will act as a brain) to build powerful ML-based Flutter applications. We will briefly touch base on the following points

1. Definitions

Let's jump to the part where most people are confused. A person who is not exposed to the IT industry might think AI, ML, & DL are all the same. So, let's understand the difference.  

‍

1.1. Artificial Intelligence (AI): 

AI, i.e. artificial intelligence, is a concept of machines being able to carry out tasks in a smarter way. You all must have used YouTube. In the search bar, you can type the lyrics of any song, even lyrics that are not necessarily the starting part of the song or title of songs, and get almost perfect results every time. This is the work of a very powerful AI.
Artificial intelligence is the ability of a machine to do tasks that are usually done by humans. This ability is special because the task we are talking about requires human intelligence and discernment.

1.2. Machine Learning (ML):

Machine learning is a subset of artificial intelligence. It is based on the idea that we expose machines to new data, which can be a complete or partial row, and let the machine decide the future output. We can also say it is a sub-field of AI that deals with the extraction of patterns from data sets. With a new data set and processing, the last result machine will slowly reach the expected result. This means that the machine can find rules for optical behavior to get new output. It also can adapt itself to new changing data just like humans.

1.3. Deep Learning (ML): 

Deep learning is again a smaller subset of machine learning, which is essentially a neural network with multiple layers. These neural networks attempt to simulate the behavior of the human brain, so you can say we are trying to create an artificial human brain. With one layer of a neural network, we can still make approximate predictions, and additional layers can help to optimize and refine for accuracy.

2. Types of ML

Before starting the implementation, we need to know the types of machine learning because it is very important to know which type is more suitable for our expected functionality.

2.1. Supervised Learning

As the name suggests, in supervised learning, the learning happens under supervision. Supervision means the data that is provided to the machine is already classified data i.e., each piece of data has fixed labels, and inputs are already mapped to the output.
Once the machine is learned, it is ready for the classification of new data.
This learning is useful for tasks like fraud detection, spam filtering, etc.

2.2. Unsupervised Learning

In unsupervised learning, the data given to machines to learn is purely raw, with no tags or labels. Here, the machine is the one that will create new classes by extracting patterns.
This learning can be used for clustering, association, etc.

2.3. Semi-Supervised Learning

Both supervised and unsupervised have their own limitations, because one requires labeled data, and the other does not, so this learning combines the behavior of both learnings, and with that, we can overcome the limitations.
In this learning, we feed row data and categorized data to the machine so it can classify the row data, and if necessary, create new clusters.

2.4. : Reinforcement Learning

For this learning, we feed the last output’s feedback with new incoming data to machines so they can learn from their mistakes. This feedback-based process will continue until the machine reaches the perfect output. This feedback is given by humans in the form of punishment or reward. This is like when a search algorithm gives you a list of results, but users do not click on other than the first result. It is like a human child who is learning from every available option and by correcting its mistakes, it grows.

3. TensorFlow

Machine learning is a complex process where we need to perform multiple activities like processing of acquiring data, training models, serving predictions, and refining future results.

To perform such operations, Google developed a framework in November 2015 called TensorFlow. All the above-mentioned processes can become easy if we use the TensorFlow framework. 

For this project, we are not going to use a complete TensorFlow framework but a small tool called TensorFlow Lite

3.1. TensorFlow Lite

TensorFlow Lite allows us to run the machine learning models on devices with limited resources, like limited RAM or memory.

3.2. TensorFlow Lite Features

• Optimized for on-device ML by addressing five key constraints: 
• Latency: because there's no round-trip to a server 
• Privacy: because no personal data leaves the device 
• Connectivity: because internet connectivity is not required 
• Size: because of a reduced model and binary size
• Power consumption: because of efficient inference and a lack of network connections
• Support for Android and iOS devices, embedded Linux, and microcontrollers
• Support for Java, Swift, Objective-C, C++, and Python programming languages
• High performance, with hardware acceleration and model optimization
• End-to-end examples for common machine learning tasks such as image classification, object detection, pose estimation, question answering, text classification, etc., on multiple platforms

4. What is Flutter?

Flutter is an open source, cross-platform development framework. With the help of Flutter by using a single code base, we can create applications for Android, iOS, web, as well as desktop. It was created by Google and uses Dart as a development language. The first stable version of Flutter was released in Apr 2018, and since then, there have been many improvements. 

5. Building an ML-Flutter Application

We are now going to build a Flutter application through which we can find the state of mind of a person from their facial expressions. The below steps explain the update we need to do for an Android-native application. For an iOS application, please refer to the links provided in the steps.

5.1. TensorFlow Lite - Native setup (Android)

• In android/app/build.gradle, add the following setting in the android block:

CODE: https://gist.github.com/velotiotech/adaed62db65503710dab2c2516a7baf5.js

5.2. TensorFlow Lite - Flutter setup (Dart)

• Create an assets folder and place your label file and model file in it. (These files we will create shortly.) In pubspec.yaml add:

 CODE: https://gist.github.com/velotiotech/2b6c4ffe504a6a0d96e983c6ceee1514.js

‍

• Run this command (Install TensorFlow Light package): 

CODE: https://gist.github.com/velotiotech/06a733724f00d2deec03e56ee942c081.js

• Add the following line to your package's pubspec.yaml (and run an implicit flutter pub get):

CODE: https://gist.github.com/velotiotech/effaf4fd68de934c3538b4201a2caf5f.js

• Now in your Dart code, you can use:

CODE: https://gist.github.com/velotiotech/3342318cfc32d75c09d406b1aa162a39.js

• Add camera dependencies to your package's pubspec.yaml (optional):

CODE: https://gist.github.com/velotiotech/44036a7c36090a4b8976e23e47ca75da.js

• Now in your Dart code, you can use:

CODE: https://gist.github.com/velotiotech/d30f53cab63d42bb9291f18176c33f50.js

• As the camera is a hardware feature, in the native code, there are few updates we need to do for both Android & iOS.  To learn more, visit:

https://pub.dev/packages/camera

• Following is the code that will appear under dependencies in pubspec.yaml once the the setup is complete.

• Flutter will automatically download the most recent version if you ignore the version number of packages.

• Do not forget to add the assets folder in the root directory.

5.3. Generate model (using website)

• Visit the following website 

• https://teachablemachine.withgoogle.com/

• Click on Get Started

‍

• Select Image project
• There are three different categories of ML projects available. We'll choose an image project since we're going to develop a project that analyzes a person's facial expression to determine their emotional condition.
• The other two types, audio project and pose project, will be useful for creating projects that involve audio operation and human pose indication, respectively.

‍

• Select Standard Image model
• Once more, there are two distinct groups of image machine learning projects. Since we are creating a project for an Android smartphone, we will select a standard picture project.
• The other type, an Embedded Image Model project, is designed for hardware with relatively little memory and computing power.

‍

• Upload images for training the classes
• We will create new classes by clicking on “Add a class.”
• We must upload photographs to these classes as we are developing a project that analyzes a person's emotional state from their facial expression.
• The more photographs we upload, the more precise our result will be.

• Click on train model and wait till training is over

• Click on Export model

• Select TensorFlow Lite Tab -> Quantized  button -> Download my model

5.4. Add files/models to the Flutter project

• Labels.txt

File contains all the class names which you created during model creation.

• *.tflite

File contains the original model file as well as associated files a ZIP.

5.5. Load & Run ML-Model

• We are importing the model from assets, so this line of code is crucial. This model will serve as the project's brain.

• Here, we're configuring the camera using a camera controller and obtaining a live feed (Cameras[0] is the front camera).

6. Conclusion

We can achieve good performance of a Flutter app with an appropriate architecture, as discussed in this blog.