Flutter, keep track of your receipts on Firebase! (Last part — back-end OCR processing using ML)

3 min readSep 7, 2021

Here we are, now this last part is for the demonstration.

First I have created a small & quick backend using spring boot, I am not going to demonstrate the whole process, but some key points are required to use Google’s API and Azure FormRecognition

I am currently using Eclipse editor for this.

We need to include some necessary groups and artefacts in Maven to get the Java packages, as shown below.

To have permission to Google Firebase, some additional resources need also to be included, especially the ServiceAccountKey which can be retrieved from the project setting you defined earlier in Firebase SDK setup and configuration.

So first you will need to initialize the service, this can be done via the @Service annotation since we keep the firebase service “online”, we are also going to use the @PostConstruct annotation as shown below.

Then a controller will allow the mobile App in Flutter to trigger the OCR process, via a GET, however, POST could be more appropriate if you wish to store resources in a back-end system.

Note this is to simplify the process, actually, it could also be a WEB application through a workflow. I will explain later how it works.

In my POC below, the OCR will be launched via the end-point /OcrDoc.

As you discover, this is going to trigger the service firebaseService.ProcessPDFOCR().

The method will go through the list of documents in Firebase having the status “Pending” (see Part II for details about what happens when you take a photo of your receipts.) and process them via RecognizerReceipts.

Cloud Firestore showing “pending” document.

See code below for some insight

The OCR processor, simple static method to do the job!

So, the line below set the modelID we have created in the previous part of this post.
String modelId = “25ccc295-cea6–451e-8598-b77e6056ae00”
The client returned will therefore analyse the form from my firebase storage URL

SyncPoller<FormRecognizerOperationResult, List<RecognizedForm>> recognizeFormPoller = client.beginRecognizeCustomFormsFromUrl(modelId, formUrl)

So let’s try it :-)

The last entry shows a receipt that I have uploaded via the mobile app in Flutter, discussed in previous posts on this long subject. The status is currently “pending”