Hi everybody,

OpenMV IDE 2.6.9 is out! It includes several bug fixes to the usability of the IDE and the latest firmware, 3.9.2!

Firmware 3.9.2 Big Features

• The latest version of ULAB has been integrated. This brings better numpy like support to MicroPython on your OpenMV Cam.
• We completely overhauled our thermal sensor support in the FIR module.

• Bilinear/Bicubic scaling is now supported for upscaling all low-resolution sensors (along with color table support, better alpha blending control, and etc. - all thanks to our new scaling pipeline).
• Support for the MLX90641 was added. The OpenMV Cam's firmware now supports the MLX90621, MLX90640, MLX90641, and AMG8833.
• And... support for the FLIR Lepton 1, 2, 2.5, 3, 3.5 has been added to the FIR module allowing you connect an external FLIR Lepton sensor to your OpenMV Cam using a Lepton breakout board.

• This allows you to do dual thermal and normal vision on any OpenMV Cam (even the old M4!).

We also completely overhauled the TV driver. It now supports the 352x240 SIF NSTC resolution at a 60 Hz update rate. Additionally, it now has all the same scaling features as the updated LCD code - like triple buffering and bicubic/bilinear scaling. This makes the TV driver significantly more useful for wireless display output.

And various bug fixes.

Here are two videos about the new features:

 

We are also continuing to integrate the new scaling pipeline into more and more functions in the firmware. to_bitmap(), to_grayscale(), to_rgb565(), to_rainbow(), copy(), crop(), scale() were all updated with the new pipeline bringing bicubic/bilinear scaling to these features along with rgb565 channel extraction and color table support.

What's next?

The LCD, FIR, and TV module were all redone to support the launch of the OpenMV Pure Thermal coming later this year. We are just waiting on the second revision of the board to come back for testing before launching the funding campaign for it. A teaser is below:

In the mean-time, we will be adding support for Embedded Display Port out over the Arduino Portenta's USB connector using the STM32H7's MIPI DSI controller next and then finally adding support for triple buffering in the camera sensor driver. Once triple buffering is integrated you will instantly see a massive boot in your algorithm FPS as the processor will no longer ever have to wait for a frame to be received. On the STM32F4 and STM32F7 based OpenMV Cams this will result in a rather high  number of interrupts per second in the 50K+ range to handle data... so, it's not going to be advised on older models. However, on the STM32H7, thanks to the MDMA hardware, we will be able to enable triple buffering of the camera data stream with nearly zero CPU load.

Anyway, that's all folks!

Hi folks,

We have a big update today. We've been doing a lot of work over the last few months and we have some major announcements. First...

OpenMV is now an official Arduino Partner!

Yes, that's right! OpenMV is now in Partnership to with Arduino to support the Arduino Portenta H7 with computer vision functionality!

This was actually announced at the beginning of last month, but, we've been way to busy to write an update email. Given that, if you want to stay up to date on any announcements we encourage you to follow us on Twitter where we post anything we're thinking about much more quickly.

Anyway, with this Partnership we'll be working closely with Arduino to improve the performance of the OpenMV library, including support in our firmware for things like Ethernet, SDIO WiFi, DisplayPort, and more!

In particular, with the Arduino Portenta H7 we are enabling audio support for using TensorFlow in MicroPython to classify sounds.

The OpenMV PureThermal

Next, besides for working with Arduino we're also Partnering with GroupGets to develop the OpenMV PureThermal board. This is the most advanced OpenMV Cam to date. It features:

• The STM32H743XI chip with 168 I/Os where we used every single one!
• An OV5640 5MP module with AutoFocus
• A FLIR Lepton 3.5 (runs in parallel with the OV5640)
• A uSD Card Slot (25 MB/s)
• 64 MB of 32-bit SDRAM at 100 MHz (400 MB/s)
• 32 MB of Quad SPI Flash (50 MB/s)
• 32 KHz Crystal for RTC operation while in deep sleep
• 1S Lipo support with USB Battery Charging
• Battery Voltage and System Current Monitoring
• WINC1500 WiFi Support (12 Mb/s)
• USB FS (12 Mb/s)
• 800x480 IPS LCD Multi-touch Screen w/ backlight dimming
• DVI Support over an HDMI connector (up to 1080p at 30 Hz)
• SparkFun QWIIC (SEEED Grove Compatible)
• RGB LED + Flash LED Support
• Buzzer Support (not shown in the picture above)
• And... the same OpenMV pin headers for expansion support

Special care was put into designing the OpenMV Pure Thermal. All external connectors on the board have TVS Diodes and we use RF chokes on high speed interfaces like the DVI output. Additionally, we made sure to tie all I/O pins on the board to the correct state when the main processor shutsdown so you can actually get the power consumption in battery mode to the sub 1mA range.

Here's a demo of the LCD working while we're doing board bringup.

Got LCD Working @groupgetscom pic.twitter.com/hi1706wjSj

— OpenMV (@openmvcam) November 9, 2020

Moving on, we plan to launch the OpenMV PureThermal at the beginning of next year. Stay tuned for more updates about the product. Please note that we've designed the OpenMV PureThermal for the needs of businesses, so, we focused on packing it with features versus trying to keep the price low. But, it should have everything you need to build your own professional Thermal Vision camera with great features.

Good Scaling Support is finally here!

For the longest we haven't had decent scaling support. However, we just integrated a new bilinear/bicubic scaling pipeline. This is actually a huge piece of code which features optimized loops for nearest neighbor, bilinear, bicubic, and area scaling for binary, grayscale, and RGB565 images. Additionally, after scaling is complete we have a tight optimized loop for drawing that scaled image using alpha, and/or an alpha lookup table, and/or a color look table, and/or rgb channel extraction, on binary, grayscale, or RGB565 images. Finally, we leverage DMA2D support where we can to handle blending in the background while the processor is scaling an image.

All of this code leads to the following performance numbers (without DMA2D enabled):

Performance Numbers (4x4 -> 128x128) - Nearest Neighbor:

BINARY draw on RGB565 - 4200 FPS
GRAYSCALE draw on RGB565 - 4500 FPS
RGB565 draw on RGB565 - 6000 FPS

BINARY draw on GRAYSCALE - 6000 FPS
GRAYSCALE draw on GRAYSCALE - 10000 FPS
RGB565 draw on GRAYSCALE - 4000 FPS

BINARY draw on BINARY- 7000 FPS
GRAYSCALE draw on BINARY - 6000 FPS
RGB565 draw on BINARY - 3400 FPS

Performance Numbers (4x4 -> 128x128) - Bilinear:

BINARY draw on RGB565 - 1310 FPS
GRAYSCALE draw on RGB565 - 1140 FPS
RGB565 draw on RGB565 - 1080 FPS

BINARY draw on GRAYSCALE - 1390 FPS
GRAYSCALE draw on GRAYSCALE - 1900 FPS
RGB565 draw on GRAYSCALE - 670 FPS

BINARY draw on BINARY- 1360 FPS
GRAYSCALE draw on BINARY - 1050 FPS
RGB565 draw on BINARY - 573 FPS

Numbers (4x4 -> 128x128) - Bicubic:

BINARY draw on RGB565 - 757 FPS
GRAYSCALE draw on RGB565 - 850 FPS
RGB565 draw on RGB565 - 535 FPS

BINARY draw on GRAYSCALE - 785 FPS
GRAYSCALE draw on GRAYSCALE - 1200 FPS
RGB565 draw on GRAYSCALE - 414 FPS

BINARY draw on BINARY- 776 FPS
GRAYSCALE draw on BINARY - 798 FPS
RGB565 draw on BINARY - 373 FPS

Which means you can scale up a grayscale 160x120 image (e.g. the FLIR Lepton 3.5 image) to 640x480 (18.75 more pixels than 128x128) and achieve:

Nearest Neighbor: 240 FPS
Bilinear: 60 FPS
Bicubic: 45 FPS

While drawing on an RGB565 image from the OV5640.

Sweet.

To get this level of performance we really dug deep into code optimization with the help of Larry Bank. Liberal use of the SIMD instructions on the Cortex-M7 made this type of performance possible. Moving forwards, we're not going to be leaving any performance on the table anymore. We're going to start leveraging as much of the STM32 hardware as we can now and write more SIMD double pumped code.

Once we have time we will be updating lens_corr() and rotation_corr() using SIMD tricks.

Best of all, note that this code works and speeds up all OpenMV Cams. Not just the H7 but also the M4 and M7.

Better LCD Support

Finally, thanks to the support of Arduino and GroupGets we're going back through our old code and massively updating the performance.

With the latest updates the LCD module now supports any resolution of SPI LCD screens with triple buffering support (if you enable this and spend the RAM) which completely decouples the process of updating the LCD screen from drawing on it. We developed triple buffering code for the 800x480 24-bit parallel LCD for the OpenMV PureThermal. But, we decided to improve our old SPI LCD support too. Additionally, since we have scaling support now we decided to integrate the new scaling pipeline into the lcd code so that you can now seamlessly scale up and down the image you're drawing on the LCD screen too.

These same improvements will also be coming to the TV module soon.

Anyway, that's all folks! Please follow us on Twitter and sponsor us on GitHub.

Hi everybody,

Just a short update, our Arduino Interface Library is finally here! You can use it to connect an OpenMV Cam to an Arduino over SoftwareSerial, HardwareSerial, SPI, I2C, and even CAN! The library includes supports for connecting your Arduino to your OpenMV Cam (or many via SPI/I2C/CAN) and also support for connecting to another Arduino (or many via SPI/I2C/CAN).

The interface library is available for download from the Arduino IDE Library Manager. You can install and play with it now.

That's all folks!