Demo Gallery
This page demonstrates four interesting image processing themes and their sub-operations. These operations are performed completely on the pixel-level within the KOVA1 (Kovilta Vision Array) chip. No image processing operations were performed on the on-board FPGA or in a computer.
These examples are just basic demonstrations of the techniques that can be applied in various application-optimized implementations. Please contact us for more information.
Real-time input dynamic range compression
With the KOVA1 pixel-level analog processing circuitry, photodiode sensor signals can be processed and evaluated even during the image integration time. This enables the implementation of image dynamic range compression in real-time, without any additional delay from e.g. multi-frame analysis or adaptation.
In the dynamic range adjustment process, the pixel integration times are limited locally if the area seen by given pixel includes very bright objects which could saturate the sensor output. In this way, all of the relevant detail in both bright and darker regions is retained. The compressive imaging mode can be applied in many situations and applications, where the illumination cannot be controlled or there input image dynamics are inherently very high.
A CD reflecting bright light, captured with a normal, global integration time mode (left) and with the real-time dynamic compression applied (right). Both video streams were read out at the same time from the KOVA1 chip, with a 1 frame difference between the normal and compressed images.
Camera moving in a scene looking both inside and out of a window, which was illuminated with direct bright sunlight.
This clip demonstrates the application of the dynamic compression mode for the imaging of an electrode welding process, with only regular ambient lighting, i.e. no additional illumination. As can be seen, the surface formations of the metal are also clearly visible, while the sensor is not saturared by the bright arc discharge and molten metal light emission. The image on the right side shows the result of a pixel-level edge detection operation.
Object segmentation and tracking with pixel-level processing
A fundamental part of the KOVA1 sensor-level processing is the segmentation of image data, which means the extraction or separation of image objects based on their spatial features or morphology. While the purpose of the grayscale front-end processing is to remove as much redundant image information as possible while retaining and emphasizing salient features, shape-based object analysis can be most efficiently performed in the 1-bit (black-and-white) image domain.
The programmable extraction and tracking of objects with 1-bit processing, based on their morphological properties, can be used to trigger the chip readout only to the presence of the desired visual objects. The image based triggering could also be used to capture a higher resolution image (or high-speed video) of the desired object with another camera controlled by the KOVA1 chip.
Pixel level processing can dramatically reduce the amount of required data transfer from the pixel array. For example, in the pipe seam feature tracking example video, the distance data (few bits per frame) would be the only output required e.g. for controlling a welding robot.
A disc with different drawn features and shapes was spinning at 3-4 revs/s. The holed square object was extracted and its path was tracked at 1000fps.
An example of on-chip triggered IO-operation, where chip readout was enabled only when the letter "A" was present in the view of the camera.
Eye tracking test, where the morphological pixel-level operations were used to extract the pupil location in real-time.
Feature tracking, where the pipe seam was extracted with pixel-level morphology. Horizontal line shows distance of seam from image center.
Spatiotemporal tracking of visual data
Because multiple image frames can be stored on the KOVA1 sensor plane, either in grayscale or binary (1-bit segmented data) form, time-domain, multi-frame analysis can also be performed on the pixel-level. Temporal events at any time-scales permitted by illumination and sensor sensitivity can be analyzed without off-chip image readout or storage of frame data outside of the sensor array.
Both the spatial and temporal properties of visual data, as well as the results of previous analysis steps, can be used to control the execution of pixel-level operations. E.g. the a processing operation or data readout can be limited to only to some parts of the image, indicated by another parallel or sequential operation. The pixel-level operations can also contain conditional branches and jumps, based on previous results obtained from the processed input data and implemented by a simple low-level controller (on chip or PCB).
This video demonstrates simple multi frame tracking of object movement (tip of the pen). The diminishing "tail" trailing the pen movement shows the position of the tip in the two previous frames.
Left: the card with the letters "L" and "R" was waved rapidly and either L or R was highlighted, depending on the movement direction of the card. Right: Readout was triggered only by movement to the left direction.
Left: hand movement was tracked in real time and the tracking data was used to reveal the result of a simulatenous edge tracking operation. Right: Activity-based image readout over multiple hours, triggered only by sufficient motion in image. The data was filtered on-chip so that e.g. changes in background illumination did not trigger readout.
Operation at all time scales
The KOVA1 chip can operate over a very large scale of temporal imaging rates. The on-chip analysis and only-on-trigger readout allow high-speed continuous tracking of visual data which, unlike conventional high-speed cameras, is not limited by the high data storage requirements. However, also extremely slow temporal events can be tracked, while still retaining all processing and image storage on the sensor level. This is made possible by static storage of frame data on the KOVA1 pixel-cell level. Even full grayscale images can be indefinitely (untill poweroff) stored on the sensor plane, thanks to a sensor-level A/D/A conversion loop.
Image capture triggered by sufficient motion. The 1-bit image shows the temporal change integrated over multiple frames that was used to trigger a new frame readout.
This video shows a scene recorded at 1kfr/s. Continuous full image data readout limits the frame-rate of the KOVA1, compared to e.g. a trigger-mode operation or only limited-data readout. However, relatively fast full-frame video recording is still possible for visualization or algorithm development purposes. The chip I/O limits the readout of grayscale image data to approximately 2000 fr/s, while 1-bit images (e.g. segmented images) can be recorded at > 10 kfr/s.
In this clip we illustrate the readout flexibility enabled by on-chip processing. The scene was continuouly tracked for any movement, i.e. a new frame was captured always when a sufficient change was detected in the scene, even if that change occurred over a very long time. This allows the same algorithm to track both the minute hand of the clock and the falling tennis ball.
Kovilta Oy
Joukahaisenkatu 1
FI-20520 Turku
Finland