Simulations
Matlab is the software package used to test algorithms. Equally, the GPL software Octave could be used to run our code. However, it is not 100% compatible yet. Making the code compatible for both Matlab and Octave should not be too difficult. If you would like assistance to make our code run on Octave, please contact us.
All data from the video decoder PCB is transmitted serially. As each pixel comes into the vision system, it is processed in a pipeline. The main stages of this pipeline are: thresholding, filtering and labelling.
Thresholding
As a starting point, we decided to investigate the use of hue to differentiate between coloured objects. Hue is independent of shading, which is great for eliminating shadows. However, objects that are greys (including black and white) create a lot of noise too. Here are two sample test images:
These are the respective hue images (saturation and intensity at maximum):
When attempting to threshold for orange, there's a lot of noise:
So we experimented with thresholding saturation and intensity, making grey become magenta:
Then combining the grey thresholding and then orange thresholding resulted with:
We want to connect a camera to an FPGA, so we will use a video decoder integrated circuit (IC) to interface between the camera and FPGA. Our camera outputs a colour composite PAL signal. The more commonly available video decoder ICs don't output HSV (Hue Saturation Value). Instead they use YCbCr. This is a Luminance (brightness) and two Chrominance (colour) components. We have now investigated the possibilities of thresholding both chrominance components and combining the results to produce a binary image similar to what we had with the HSV colour space.
These are the thresholded Cb components:
These are the Cr components:
Combining the results together:
From these pictures we can see that using both Chrominance parts of a YCbCr image gives better results than thresholding the Hue of an image in the HSV colour space.
Binary Image Algorithms
After thresholding has been done, the binary images must be processed to determine which pixels are connected to form objects.
One method involves scanning through the binary image and comparing each pixel with it's neigbours to see if they form part of an object. Connected pixels can be grouped together in a labelling procedure.
This task becomes difficult because pixels in the same object may be labelled differently. Then it becomes a matter of checking which labels are connected together.
Image Processing
Pixels are processed in a pipeline. Pixels are thresholded for colour, then labeled according to the label of its neigbours. If there are no neigbouring pixels, a new label is issued. Below is an example binary image achieved after thresholding:
Each pixel is read sequentially from left to right, then down each row. The labelling algorithm is to compare the three pixels above and the pixel on the immediate left. If neigbouring labels exist, the label with the smallest value is given to the target pixel.
The target pixel is labelled as green(2) because compared to blue(3) it has the lower value.
This is the result of the labelling process. We can see that the three distinct objects have been labelled with 9 labels. The number of labels will now be reduced down to the number of objects.
A relationship table is produced that shows how labels are linked together. Scanning down each column will find labels that touch the selected label. For example, in column 1, label 4 is connected to label 1.
The sorting algorithm works as follows:
- Scan down column to find each connected row.
- For each connected row, perform a Logical OR operation and place result in the highest connected row.
- Delete the contents of all connected rows except the highest one.
- Repeat for each column
- Shift rows to the top.
The sorted relationship table can now be used to interpret the 9-label image.Each row is an object label that maps to the original 9 labels via the columns.
