How to Identify the Text in an Image Using Firebase ML Kit?
Introduction
Are you an Android developer and want to start a career in mobile application through Machine Learning (ML)? Well, you're in the right place. Check out the previous article to learn more about ML and why it's a next major thing? Or, why mobile app developers should start implementing it on their apps?
We have chosen Firebase ML Kit to incorporate with ML algorithms for mobile apps and here's a basic introduction on coding.
Ever clicked a picture or captured a screenshot with a lot of texts and faced a problem in typing? The reason for this is after some week photo filters and captions rolls out.
After reading this article, you'll be able to make an app of your own and try modifying the text from the chosen image.
List of a resource before you start:
1. Android Studio v3.0+
2. Testing device
3. Basic knowledge in Android development in Java
4. Basic understanding of Machine Learning
Let's get started!
Just follow these steps:
1. Go to the Firebase Console
2. Click on "Create New Project" and name it
Connect your Android app
1. From the overview screen of your new project, click "Add Firebase" to your Android app.
2. Enter the codelab's package name: com.google.firebase.codelab.mlkit.
3. Leave the other fields blank and click Register app.
4. Add google-services.json file to your app:
After adding the package name and select Continue to download the configuration file automatically. The file contains all the Firebase metadata that supports the application. Copy the google-services.json file on the app directory and skip the instructions to add the Firebase SDK on your app.
Add the dependencies:
build.gradle
dependencies {
// ...
implementation 'com.google.firebase:firebase-ml-vision:18.0.1'
implementation 'com.google.firebase:firebase-ml-vision-image-label-model:17.0.2'
implementation 'com.google.firebase:firebase-ml-vision-face-model:17.0.2'
implementation 'com.google.firebase:firebase-ml-model-interpreter:16.2.3'
}
apply plugin: 'com.google.gms.google-services'
Once this is done, just sync your project with the gradle file.
Add on-device text recognition:
In this step, add functionality to your app to recognize text in images.
Add following lines of code in your activity to configure the text recognition detector:
FirebaseVisionImage image = FirebaseVisionImage.fromBitmap(mSelectedImage);
FirebaseVisionTextRecognizer recognizer = FirebaseVision.getInstance()
.getOnDeviceTextRecognizer();
mTextButton.setEnabled(false);
recognizer.processImage(image)
.addOnSuccessListener(
new OnSuccessListener() {
@Override
public void onSuccess(FirebaseVisionText texts) {
mTextButton.setEnabled(true);
processTextRecognitionResult(texts);
}
})
.addOnFailureListener(
new OnFailureListener() {
@Override
public void onFailure(@NonNull Exception e) {
// Task failed with an exception
mTextButton.setEnabled(true);
e.printStackTrace();
}
});
Process the text recognition response:
Add the following code to processTextRecognitionResult in the activity class to parse the results and display them in your app.
private void processTextRecognitionResult(FirebaseVisionText texts) {
List blocks = texts.getTextBlocks();
if (blocks.size() == 0) {
showToast("No text found");
return;
}
mGraphicOverlay.clear();
for (int i = 0; i < blocks.size(); i++) {
List lines = blocks.get(i).getLines();
for (int j = 0; j < lines.size(); j++) {
List elements = lines.get(j).getElements();
for (int k = 0; k < elements.size(); k++) {
Graphic textGraphic = new TextGraphic(mGraphicOverlay, elements.get(k));
mGraphicOverlay.add(textGraphic);
}
}
}
}
That's all. Just run your app and start playing with it!
Conclusion
This was an implementation of Firebase ML kit for Android applications to detect the text from an image. Firebase SDK is compatible with iOS device the implementation is under BETA stage. It has many features, such as detection of a face, barcode, landmark, and text also image labelling.
However, you can deploy on your own models in Firebase using TensorFlow Lite. Apart from this, Google Cloud Vision and Neural Network API on a single SDK. The API works on both Device and Cloud as well.
At Appiness, we provide app development service to create a custom mobile applications and also specialize in developing apps for iOS and Android. We offer a wide range of services including UI/UX design, app testing, and ongoing maintenance. Contact us today to get started.