Automatic Tool for Cropping Bills from Scans

What Is It?

At work, we’re developing a project aimed at small and medium-sized businesses that goes beyond simple accounting. It includes many innovative features, but today I want to focus on one in particular: the automatic processing of bills and invoices. Users can take advantage of this feature via a mobile client (Android, iOS) or a web application.

The Challenge

We encountered an issue where users were scanning multiple bills together in a single image, rather than following the "one image - one document" rule. This made it difficult to process the documents correctly. To address this, I developed a script that automatically detects and crops these multi-bill images into separate, individual images.

The Algorithm

To separate and crop the bills from a scanned image, I used a series of simple image preprocessing methods:

  1. Resize: The image is resized to a standard dimension for consistency.
  2. Morphological Operation (IMOPEN): This step helps in removing noise and refining the structure of the image.
  3. Convert to Binary: The image is converted into a binary format, where pixels are categorized as either black or white, making it easier to identify distinct regions.
  4. Blob Detection: The binary image is analyzed to detect continuous white areas (blobs), which represent the bills.
  5. Find Largest Blob: The largest blobs are identified as potential bill areas.
  6. Bounding Box: A bounding box is drawn around each detected blob to define the region of interest.
  7. Percentage Limit: This step ensures that only sufficiently large blobs (likely to be bills) are considered.
  8. Crop and Resize: The identified regions are cropped out and resized as individual images.

The code implementing these steps is straightforward and self-explanatory.

Requirements

To run this script, you’ll need:

  • GNU Octave, version 3.6.4
  • Octave with the image package
       
#! /bin/octave -qf
# a sample Octave program

% @Author Miroslav Bodis
% created: 2014-10-04
% updated: 2015-10-04

% -- -- CUT BILL/BILLS FROM PICTURE -- --
% 0. VALIDATE INPUT
% 1. RESIZE
% 2. IMOPEN - MORPHOLOGICAL OPERATION
% 4. IMG TO BINARY
% 5. BLOD DETECTS 
% 6. FIND BIGGEST BLOB
% 7. GET BOUNDING BOX
% 8. PERCENTAGE LIMIT
% 9. CROP WITH RESIZE



IMG_TO_BINARY_TRASH = 0.3;
IMG_LIMIT_BILL_DETECT = 1000;
IMG_LIMIT_BILL_PERCENT = 15;
IMG_SIZE_LIMIT = 550;
DEBUG = false;
% DEBUG = true;




% 0 ---- VALIDATE INPUT
%------------------------------
% - input require input image
 arg_list = argv();

 if ( size(arg_list, 1) == 0)
   printf("\nrequired 1 arg, input file \n");
   return;
 endif;  

 %validate input
 if ( size(arg_list, 1) != 1)
     printf("\nERROR invalid input, allowing only one image as input_file! \n\n");
   return;
 endif;

 %input exists
 if (!exist(arg_list{1}))
   fprintf("\n image %s not exists\n", arg_list{1});
   return;
 endif;

 pkg load image;



% 1 ---- RESIZE 
%------------------------------
 img_rgb_full = imread(arg_list{1}); 

 [height, width, rgb] = size(img_rgb_full);
 resize = 1;
 if (width > IMG_SIZE_LIMIT && width > height)
  resize = IMG_SIZE_LIMIT / width;
 elseif (height > IMG_SIZE_LIMIT)
  resize = IMG_SIZE_LIMIT / height;
 endif;

 img_rgb = imresize(img_rgb_full, resize, 'nearest');
 if (DEBUG)
  imwrite(img_rgb, 'debug_0_resize.jpg', 'jpg', 'Quality', 100);
 endif;



% 2 ---- IMOPEN - MORPHOLOGICAL OPERATION
%------------------------------
 gray = rgb2gray(img_rgb);

 % disc = uint8 ([0 0 0 1 0 0 0
 %               0 1 1 1 1 1 0
 %               0 1 1 1 1 1 0
 %               1 1 1 1 1 1 1
 %               0 1 1 1 1 1 0
 %               0 1 1 1 1 1 0
 %               0 0 0 1 0 0 0]);

 disc = uint8 ([0 0 0 0 0 1 0 0 0 0 0
      0 0 0 1 1 1 1 1 0 0 0
      0 0 1 1 1 1 1 1 1 0 0
      0 1 1 1 1 1 1 1 1 1 0 
      0 1 1 1 1 1 1 1 1 1 0
      1 1 1 1 1 1 1 1 1 1 1
      0 1 1 1 1 1 1 1 1 1 0
      0 1 1 1 1 1 1 1 1 1 0
      0 0 1 1 1 1 1 1 1 0 0
      0 0 0 1 1 1 1 1 0 0 0
      0 0 0 0 0 1 0 0 0 0 0]);

 gray = imdilate (imerode(gray, disc), disc);
 if (DEBUG)
  imwrite(gray, 'debug_1_opening.jpg', 'jpg', 'Quality', 100);
 endif;


% 3 ---- IMG TO BINARY
%------------------------------
 bw = im2bw(gray, IMG_TO_BINARY_TRASH);
 if (DEBUG)
  imwrite(bw, 'debug_2_im2bw.jpg', 'jpg', 'Quality', 100);
 endif;



% 4 ---- BLOB DETECTS 
%------------------------------
 cc = bwconncomp(bw, 4);
 % cc.NumObjects % print number of blobs
 % cc.ImageSize % print image size

 % -- -- select blob number 225
 % grain = false(size(bw));
 % grain(cc.PixelIdxList{225}) = true;
 % imwrite(grain, 'output.jpg', 'jpg', 'Quality', 100);



% 5 ---- FIND BIGGEST BLOB 
%------------------------------
 % -- -- biggest blob
 % numPixels = cellfun(@numel,cc.PixelIdxList); 
 % [biggest,idx] = max(numPixels);   

 % -- find more bigger objects
 object = 0;
 for j = 1:cc.NumObjects  
  
  [s1,s2] = size(cc.PixelIdxList{j});

  if (s1 > IMG_LIMIT_BILL_DETECT)
   object+=1;   
   bw = false(size(bw));    % black img   
   bw(cc.PixelIdxList{j}) = true; % add selected object  
   if (DEBUG)
    imwrite(bw, 'debug_34_biggest_bloc.jpg', 'jpg', 'Quality', 100);
   endif;



% 6 ---- GET BOUNDING BOX
%------------------------------
   boundaries = bwboundaries(bw);
   numberOfBoundaries = size(boundaries);

   from = boundaries{1};
   fromx = min(from(:,1));
   fromy = min(from(:,2));

   to = boundaries{2};
   tox = max(to(:,1));
   toy = max(to(:,2));


   if (DEBUG)
    for k = 1 : numberOfBoundaries
        thisBoundary = boundaries{k};
        plot(thisBoundary(:,2), thisBoundary(:,1), 'g', 'LineWidth', 2); 
    end  
    print("debug_5_boundbox_plot.png", "-dpng");
   endif;



% 7 ---- PERCENTAGE LIMIT
%------------------------------
   l=tox-fromx;
   w=toy-fromy;
   [x,y,rgb] = size(img_rgb);
   blob_percent_area = ((l*w) / (x*y) *100)
   if ( blob_percent_area > IMG_LIMIT_BILL_PERCENT)  

% 8 ---- CROP WITH RESIZE 
%------------------------------
   extra = (1/resize);
   x1 = round( fromx * extra);
   x2 = round( (x1 + l * extra) );
   y1 = round( fromy * extra );
   y2 = round( (y1 + w * extra) );

   crop_img = img_rgb_full(x1:x2, y1:y2);

    name = char ([98, 105, 108, 108, 45, 48+object]); % bill-1, bill-2 ... 
    imwrite(crop_img, name, 'jpg', 'Quality', 100);

   endif; 



  endif; 
 endfor

        
         

Comments

Post a Comment

Popular posts from this blog

Skate Tricks Recognition Using Gyroscope

Image
In this article, we’ll describe how to recognize skateboard tricks using a gyroscope. This sensor is already present in most smartphones, but if you’re not familiar with it, here is a description . Before we dive in, a small disclaimer: This project originated as part of a Slido hackathon with the goal of demonstrating that it’s possible to recognize skateboard tricks using a gyroscope. For simplicity, we’ll focus on recognizing just two basic tricks and won’t consider the skater’s stance on the board. We’ll break down the problem into several smaller tasks: recording the trick, storing the data, describing the trick, analyzing it, and, finally, recognizing it. Recording the Trick Let’s start with trick recording. To obtain precise data, the device with the gyroscope needs to be attached directly to the skateboard. Place it on the bottom of the board to avoid interfering with the execution of the trick. The device should also be compact and lightweight to avoid altering the board’s cen...
Read more

Play table

Image
Using proximity sensors for playing midi tones combined with LED visualization Description The goal of this project was to create a table-sized device equipped with multiple proximity sensors capable of playing MIDI tones. Each sensor is accompanied by LEDs that indicate the distance of the user's hand above the table. This interactive table can be used by one or more people simultaneously. Hardware Setup I began with a cardboard prototype to test the sensors and the underlying logic. Afterward, I ordered a custom plottered sticker with a design which was painted with bare conductive paint. I drilled holes and connected the touchboard to seven Arduino Nanos, each controlling 13 LEDs. Additionally, I incorporated two potentiometers: one for volume control and another for changing the note setup. Programming The programming task was divided into two parts: Master Program (Touchboard): Reads values from the proximity sensors and sends messages to the slave Arduinos. Sl...
Read more

Counting dice and train wagons using computer vision

Image
Computer vision exercises with preprocessing Before starting my next project, I decided to work on some computer vision exercises. Each example is based on straightforward image preprocessing techniques. No complex data structures or machine learning are involved. Dice Detection I got this idea while browsing the net and became curious about how challenging it would be to write such a script. Here’s a step-by-step breakdown of the algorithm: Movement Detection : By comparing several frames with thresholds, we can determine if there is any movement in the frame. Adding a small time buffer after the movement stops gives us more accurate information. Removing the Background : Thresholding the grayscale frame helps to remove the background, leaving us with a binary image that highlights the objects. Cropping the Objects : Using contours, we can detect and isolate the objects by cropping them. Detecting Dots : By inverting the image, the dots on the dice become more distinguish...
Read more