Hi folks,
We have a big update today. We've been doing a lot of work over the last few months and we have some major announcements. First...
OpenMV is now an official Arduino Partner!
Yes, that's right! OpenMV is now in Partnership to with Arduino to support the Arduino Portenta H7 with computer vision functionality!
This was actually announced at the beginning of last month, but, we've been way to busy to write an update email. Given that, if you want to stay up to date on any announcements we encourage you to follow us on Twitter where we post anything we're thinking about much more quickly.
Anyway, with this Partnership we'll be working closely with Arduino to improve the performance of the OpenMV library, including support in our firmware for things like Ethernet, SDIO WiFi, DisplayPort, and more!
In particular, with the Arduino Portenta H7 we are enabling audio support for using TensorFlow in MicroPython to classify sounds.
The OpenMV PureThermal
Next, besides for working with Arduino we're also Partnering with GroupGets to develop the OpenMV PureThermal board. This is the most advanced OpenMV Cam to date. It features:
- The STM32H743XI chip with 168 I/Os where we used every single one!
- An OV5640 5MP module with AutoFocus
- A FLIR Lepton 3.5 (runs in parallel with the OV5640)
- A uSD Card Slot (25 MB/s)
- 64 MB of 32-bit SDRAM at 100 MHz (400 MB/s)
- 32 MB of Quad SPI Flash (50 MB/s)
- 32 KHz Crystal for RTC operation while in deep sleep
- 1S Lipo support with USB Battery Charging
- Battery Voltage and System Current Monitoring
- WINC1500 WiFi Support (12 Mb/s)
- USB FS (12 Mb/s)
- 800x480 IPS LCD Multi-touch Screen w/ backlight dimming
- DVI Support over an HDMI connector (up to 1080p at 30 Hz)
- SparkFun QWIIC (SEEED Grove Compatible)
- RGB LED + Flash LED Support
- Buzzer Support (not shown in the picture above)
- And... the same OpenMV pin headers for expansion support
Special care was put into designing the OpenMV Pure Thermal. All external connectors on the board have TVS Diodes and we use RF chokes on high speed interfaces like the DVI output. Additionally, we made sure to tie all I/O pins on the board to the correct state when the main processor shutsdown so you can actually get the power consumption in battery mode to the sub 1mA range.
Here's a demo of the LCD working while we're doing board bringup.
Got LCD Working @groupgetscom pic.twitter.com/hi1706wjSj
— OpenMV (@openmvcam) November 9, 2020
Moving on, we plan to launch the OpenMV PureThermal at the beginning of next year. Stay tuned for more updates about the product. Please note that we've designed the OpenMV PureThermal for the needs of businesses, so, we focused on packing it with features versus trying to keep the price low. But, it should have everything you need to build your own professional Thermal Vision camera with great features.
Good Scaling Support is finally here!
For the longest we haven't had decent scaling support. However, we just integrated a new bilinear/bicubic scaling pipeline. This is actually a huge piece of code which features optimized loops for nearest neighbor, bilinear, bicubic, and area scaling for binary, grayscale, and RGB565 images. Additionally, after scaling is complete we have a tight optimized loop for drawing that scaled image using alpha, and/or an alpha lookup table, and/or a color look table, and/or rgb channel extraction, on binary, grayscale, or RGB565 images. Finally, we leverage DMA2D support where we can to handle blending in the background while the processor is scaling an image.
All of this code leads to the following performance numbers (without DMA2D enabled):
Performance Numbers (4x4 -> 128x128) - Nearest Neighbor:
BINARY draw on RGB565 - 4200 FPS
GRAYSCALE draw on RGB565 - 4500 FPS
RGB565 draw on RGB565 - 6000 FPS
BINARY draw on GRAYSCALE - 6000 FPS
GRAYSCALE draw on GRAYSCALE - 10000 FPS
RGB565 draw on GRAYSCALE - 4000 FPS
BINARY draw on BINARY- 7000 FPS
GRAYSCALE draw on BINARY - 6000 FPS
RGB565 draw on BINARY - 3400 FPS
Performance Numbers (4x4 -> 128x128) - Bilinear:
BINARY draw on RGB565 - 1310 FPS
GRAYSCALE draw on RGB565 - 1140 FPS
RGB565 draw on RGB565 - 1080 FPS
BINARY draw on GRAYSCALE - 1390 FPS
GRAYSCALE draw on GRAYSCALE - 1900 FPS
RGB565 draw on GRAYSCALE - 670 FPS
BINARY draw on BINARY- 1360 FPS
GRAYSCALE draw on BINARY - 1050 FPS
RGB565 draw on BINARY - 573 FPS
Numbers (4x4 -> 128x128) - Bicubic:
BINARY draw on RGB565 - 757 FPS
GRAYSCALE draw on RGB565 - 850 FPS
RGB565 draw on RGB565 - 535 FPS
BINARY draw on GRAYSCALE - 785 FPS
GRAYSCALE draw on GRAYSCALE - 1200 FPS
RGB565 draw on GRAYSCALE - 414 FPS
BINARY draw on BINARY- 776 FPS
GRAYSCALE draw on BINARY - 798 FPS
RGB565 draw on BINARY - 373 FPS
Which means you can scale up a grayscale 160x120 image (e.g. the FLIR Lepton 3.5 image) to 640x480 (18.75 more pixels than 128x128) and achieve:
Nearest Neighbor: 240 FPS
Bilinear: 60 FPS
Bicubic: 45 FPS
While drawing on an RGB565 image from the OV5640.
Sweet.
To get this level of performance we really dug deep into code optimization with the help of Larry Bank. Liberal use of the SIMD instructions on the Cortex-M7 made this type of performance possible. Moving forwards, we're not going to be leaving any performance on the table anymore. We're going to start leveraging as much of the STM32 hardware as we can now and write more SIMD double pumped code.
Once we have time we will be updating lens_corr() and rotation_corr() using SIMD tricks.
Best of all, note that this code works and speeds up all OpenMV Cams. Not just the H7 but also the M4 and M7.
Better LCD Support
Finally, thanks to the support of Arduino and GroupGets we're going back through our old code and massively updating the performance.
With the latest updates the LCD module now supports any resolution of SPI LCD screens with triple buffering support (if you enable this and spend the RAM) which completely decouples the process of updating the LCD screen from drawing on it. We developed triple buffering code for the 800x480 24-bit parallel LCD for the OpenMV PureThermal. But, we decided to improve our old SPI LCD support too. Additionally, since we have scaling support now we decided to integrate the new scaling pipeline into the lcd code so that you can now seamlessly scale up and down the image you're drawing on the LCD screen too.
These same improvements will also be coming to the TV module soon.
Anyway, that's all folks! Please follow us on Twitter and sponsor us on GitHub.
