Advertisements

Saddest Arduino Demo

The Adafruit 2.8 inch TFT Touch Shield for Arduino v2 seems just about ideal for a small control panel, such as one might use with a modified sewing machine. All you need is a few on-screen buttons, a few status display, and a bit of Arduino love: what more could one ask?

So I gimmicked up some small buttons with GIMP, made two large buttons with ImageMagick, and lashed together some Arduino code based on the Adafruit demo:

Adafruit TFT display - timing demo

Adafruit TFT display – timing demo

The picture doesn’t do justice to the display: it’s a nice piece of hardware that produces a crisp image. The moire patterns come from the interaction of TFT display pixels / camera pixels / image resizing.

It’s not obvious from the Adafruit description, but the display is inherently portrait-mode as shown. The (0,0) origin lies in the upper left corner, just over the DC power jack, and screen buffer updates proceed left-to-right, top-to-bottom from there.

The gotcha: even with the Arduino Mega’s hardware SPI, writing the full display requires nearly 4 seconds. Yeah, slow-scan TV in action.  Writing the screen with a solid color requires several seconds.

After commenting out the serial tracing instructions from the Adafruit demo and tweaking a few other things, these timings apply:

-------------
Writing background
Elapsed: 3687
Writing six buttons
Elapsed: 529
Overwriting six buttons
Elapsed: 531
Rewriting buttons 10 times
Elapsed: 1767
Overwriting 2 large buttons
Elapsed: 1718

The timings work out to:

  • Background: 240×320 → 48 µs/pixel
  • Smaller buttons: 50×25 → 71 µs/pixel → 88 ms/button
  • Rewriting one button: 71 µs/pixel
  • Larger buttons: 120×64 → 56 µs/pixel → 430 ms/button

The button images come from BMP files on a MicroSD card and 8 KB of RAM won’t suffice for even a small button. Instead, the transfer loop buffers 20 pixels = 60 bytes from the card, writes them to the display, and iterates until it’s done.

Trust me on this: watching the buttons gradually change is depressing.

Yes, it could work as a control panel for the sewing machine, but it doesn’t have nearly the pep we’ve come to expect from touch-screen gadgetry. On the other paw, Arduinos are 8-bit microcontrollers teleported from the mid-90s, when crappy 20×4 LCD panels were pretty much as good as it got.

The SPI hardware clock runs at half the oscillator frequency = 16 MHz/2 = 8 MHz = 125 ns/bit. Clocking a single byte thus requires 1 µs; even allowing that each pixel = 3 bytes = 6 SPI operations, there’s a lot of software overhead ripe for the chopping block. I’d be sorely tempted to write a pair of unrolled loops that read 20 pixels and write 20 pixels with little more than a pointer increment & status spin between successive SPI transfers.

It’ll make hotshot C++ programmers flinch, but splicing all that into a single routine, throwing out all the clipping and error checking, and just getting it done might push the times down around 20 µs/pixel. Admittedly, that’s barely twice as fast, but should be less depressing.

However, even with all that effort, the time required to update the screen will clobber the motor control. You can’t devote a big fraction of a second to rewriting a button at the same time you’re monitoring the pedal position, measuring the motor speed, and updating the control voltage. That’s the same problem that makes Arduino microcontrollers inadequate for contemporary 3D printers: beyond a certain point, the hardware is fighting your efforts, not helping you get things done.

A reasonable workaround: no screen updates while the motor turns, because you shouldn’t have any hands free while you’re sewing.

There’s also no way to make up for not having enough RAM or video hardware. The same display runs full-speed video on a Raspberry Pi…

The Arduino source code, hacked from the Adafruit demo:

// 2.8 inch TFT display exercise
// Crudely hacked from Adafruit demo code: spitftbitmap.ino
// 2014-07-03 Ed Nisley KE4ZNU

/***************************************************
  This is our Bitmap drawing example for the Adafruit ILI9341 Breakout and Shield
  ----> http://www.adafruit.com/products/1651

  Check out the links above for our tutorials and wiring diagrams
  These displays use SPI to communicate, 4 or 5 pins are required to
  interface (RST is optional)
  Adafruit invests time and resources providing this open source code,
  please support Adafruit and open-source hardware by purchasing
  products from Adafruit!

  Written by Limor Fried/Ladyada for Adafruit Industries.
  MIT license, all text above must be included in any redistribution
 ****************************************************/

#include <Adafruit_GFX.h>    // Core graphics library
#include "Adafruit_ILI9341.h" // Hardware-specific library
#include <SPI.h>
#include <SD.h>

#define TFT_DC 9
#define TFT_CS 10
Adafruit_ILI9341 tft = Adafruit_ILI9341(TFT_CS, TFT_DC);

#define SD_CS 4

unsigned long MillisNow, MillisThen;

void setup(void) {
  Serial.begin(115200);

  tft.begin();
  tft.fillScreen(ILI9341_BLACK);

  Serial.print(F("Initializing SD card..."));
  if (!SD.begin(SD_CS)) {
    Serial.println(F("failed!"));
  }
  Serial.println(F("OK!"));

}

void loop() {
  Serial.println(F("--------------"));
  Serial.println(F("Writing background"));
  MillisThen = millis();

  bmpDraw("Test1.bmp", 0, 0);

  MillisNow = millis();
  Serial.print(F("" Elapsed: ""));
  Serial.println(MillisNow - MillisThen);

  Serial.println(F("Writing 6 small buttons"));
  MillisThen = millis();

  bmpDraw("Red50x25.bmp",10,10);
  bmpDraw("Red50x25.bmp",10,50);
  bmpDraw("Red50x25.bmp",10,100);

  bmpDraw("Grn50x25.bmp",80,25);
  bmpDraw("Grn50x25.bmp",80,75);
  bmpDraw("Grn50x25.bmp",80,125);

  MillisNow = millis();
  Serial.print(F("" Elapsed: ""));
  Serial.println(MillisNow - MillisThen);

  Serial.println(F("Overwriting 6 small buttons"));
  MillisThen = millis();

  bmpDraw("Grn50x25.bmp",10,10);
  bmpDraw("Grn50x25.bmp",10,50);
  bmpDraw("Grn50x25.bmp",10,100);

  bmpDraw("Red50x25.bmp",80,25);
  bmpDraw("Red50x25.bmp",80,75);
  bmpDraw("Red50x25.bmp",80,125);

  MillisNow = millis();
  Serial.print(F("" Elapsed: ""));
  Serial.println(MillisNow - MillisThen);

  Serial.println(F("Writing small button 10x2 times"));
  MillisThen = millis();

  for (byte i=0; i<10; i++) {
    bmpDraw("Grn50x25.bmp",10,175);
    bmpDraw("Red50x25.bmp",10,175);
  }

  MillisNow = millis();
  Serial.print(F("" Elapsed: ""));
  Serial.println(MillisNow - MillisThen);

  Serial.println(F("Overwriting 2 large buttons"));
  MillisThen = millis();  

  bmpDraw("GelDn.bmp",0,250);
  bmpDraw("GelUp.bmp",0,250);

  bmpDraw("GelUp.bmp",120,250);
  bmpDraw("GelDn.bmp",120,250);

  MillisNow = millis();
  Serial.print(F("" Elapsed: ""));
  Serial.println(MillisNow - MillisThen);

}

// This function opens a Windows Bitmap (BMP) file and
// displays it at the given coordinates.  It's sped up
// by reading many pixels worth of data at a time
// (rather than pixel by pixel).  Increasing the buffer
// size takes more of the Arduino's precious RAM but
// makes loading a little faster.  20 pixels seems a
// good balance.

#define BUFFPIXEL 20

void bmpDraw(char *filename, uint8_t x, uint16_t y) {

  File     bmpFile;
  int      bmpWidth, bmpHeight;   // W+H in pixels
  uint8_t  bmpDepth;              // Bit depth (currently must be 24)
  uint32_t bmpImageoffset;        // Start of image data in file
  uint32_t rowSize;               // Not always = bmpWidth; may have padding
  uint8_t  sdbuffer[3*BUFFPIXEL]; // pixel buffer (R+G+B per pixel)
  uint8_t  buffidx = sizeof(sdbuffer); // Current position in sdbuffer
  boolean  goodBmp = false;       // Set to true on valid header parse
  boolean  flip    = true;        // BMP is stored bottom-to-top
  int      w, h, row, col;
  uint8_t  r, g, b;
  uint32_t pos = 0, startTime = millis();

  if((x >= tft.width()) || (y >= tft.height())) return;

//  Serial.println();
//  Serial.print(F("Loading image '"));
//  Serial.print(filename);
//  Serial.println('\'');

  // Open requested file on SD card
  if ((bmpFile = SD.open(filename)) == NULL) {
//    Serial.print(F("File not found"));
    return;
  }

  // Parse BMP header
  if(read16(bmpFile) == 0x4D42) { // BMP signature
//    Serial.print(F("File size: "));
//	Serial.println(read32(bmpFile));
	read32(bmpFile);
    (void)read32(bmpFile); // Read & ignore creator bytes
    bmpImageoffset = read32(bmpFile); // Start of image data
//    Serial.print(F("Image Offset: ")); Serial.println(bmpImageoffset, DEC);
    // Read DIB header
//    Serial.print(F("Header size: "));
//	Serial.println(read32(bmpFile));
	read32(bmpFile);
    bmpWidth  = read32(bmpFile);
    bmpHeight = read32(bmpFile);
    if(read16(bmpFile) == 1) { // # planes -- must be '1'
      bmpDepth = read16(bmpFile); // bits per pixel
//      Serial.print(F("Bit Depth: ")); Serial.println(bmpDepth);
      if((bmpDepth == 24) && (read32(bmpFile) == 0)) { // 0 = uncompressed

        goodBmp = true; // Supported BMP format -- proceed!
//        Serial.print(F("Image size: "));
//        Serial.print(bmpWidth);
//        Serial.print('x');
//        Serial.println(bmpHeight);

        // BMP rows are padded (if needed) to 4-byte boundary
        rowSize = (bmpWidth * 3 + 3) & ~3;

        // If bmpHeight is negative, image is in top-down order.
        // This is not canon but has been observed in the wild.
        if(bmpHeight < 0) {
          bmpHeight = -bmpHeight;
          flip      = false;
        }

        // Crop area to be loaded
        w = bmpWidth;
        h = bmpHeight;
        if((x+w-1) >= tft.width())  w = tft.width()  - x;
        if((y+h-1) >= tft.height()) h = tft.height() - y;

        // Set TFT address window to clipped image bounds
        tft.setAddrWindow(x, y, x+w-1, y+h-1);

        for (row=0; row<h; row++) { // For each scanline...

          // Seek to start of scan line.  It might seem labor-
          // intensive to be doing this on every line, but this
          // method covers a lot of gritty details like cropping
          // and scanline padding.  Also, the seek only takes
          // place if the file position actually needs to change
          // (avoids a lot of cluster math in SD library).
          if(flip) // Bitmap is stored bottom-to-top order (normal BMP)
            pos = bmpImageoffset + (bmpHeight - 1 - row) * rowSize;
          else     // Bitmap is stored top-to-bottom
            pos = bmpImageoffset + row * rowSize;
          if(bmpFile.position() != pos) { // Need seek?
            bmpFile.seek(pos);
            buffidx = sizeof(sdbuffer); // Force buffer reload
          }

          for (col=0; col<w; col++) { // For each pixel...
            // Time to read more pixel data?
            if (buffidx >= sizeof(sdbuffer)) { // Indeed
              bmpFile.read(sdbuffer, sizeof(sdbuffer));
              buffidx = 0; // Set index to beginning
            }

            // Convert pixel from BMP to TFT format, push to display
            b = sdbuffer[buffidx++];
            g = sdbuffer[buffidx++];
            r = sdbuffer[buffidx++];
            tft.pushColor(tft.color565(r,g,b));
          } // end pixel
        } // end scanline
//        Serial.print(F("Loaded in "));
//        Serial.print(millis() - startTime);
//        Serial.println("" ms"");
      } // end goodBmp
    }
  }

  bmpFile.close();
  if(!goodBmp) Serial.println(F("BMP format not recognized."));
}

// These read 16- and 32-bit types from the SD card file.
// BMP data is stored little-endian, Arduino is little-endian too.
// May need to reverse subscript order if porting elsewhere.

uint16_t read16(File &f) {
  uint16_t result;
  ((uint8_t *)&result)[0] = f.read(); // LSB
  ((uint8_t *)&result)[1] = f.read(); // MSB
  return result;
}

uint32_t read32(File &f) {
  uint32_t result;
  ((uint8_t *)&result)[0] = f.read(); // LSB
  ((uint8_t *)&result)[1] = f.read();
  ((uint8_t *)&result)[2] = f.read();
  ((uint8_t *)&result)[3] = f.read(); // MSB
  return result;
}
Advertisements

,

  1. #1 by Jason Doege on 2014-07-28 - 08:24

    You really ought to look at the Beaglebone Black as your go-to automation gadget. It is really quite amazing and for the real-time stuff you are into with CNC and 3-D printing, there is a dedicated real-time processor that can be used to control the IOs.

    • #2 by Ed on 2014-07-30 - 14:14

      I have the uneasy feeling this will turn into a BBB project, but all I really wanted was a quick-and-easy display; so much for that.

      Most likely, I’ll go with a few LEDs or a 4×16 LCD until I get the motor control sorted out, then I can complexicate the UI beyond recognition.

  2. #3 by macca1945 on 2014-07-28 - 09:23

    Have a look at the 4D Systems screens, they play nicely with Arduino, but a little expensive.

    • #4 by Ed on 2014-07-30 - 14:16

      That looks like a lot more hammer than I really need, but thanks for the hint: it may come in handy for something around here!

  3. #5 by Red County Pete on 2014-07-28 - 09:39

    One of my projects is computer control for a telescope drive. Years ago, I took some GPL’ed source code from BBAstrodesigns and ported it from DOS to Linux, but the code is based on out of date hardware, with parallel and serial ports. I never got the courage to try the real-time addons with the only Linux box I had at the time. I now have another, but most of the old code is a dead end.

    I’ve figured that I could eliminate/reduce the problems by letting a laptop do the GUI and tell motor controllers what to do via USB, with one or two Arduinos handling the motor drives.

    • #6 by Ed on 2014-07-30 - 14:19

      In this day & age, a Beaglebone Black running LinuxCNC with a motor driver cape might be exactly the right hammer for the job: hardware pulse generation, good UI tools, and you could probably do the entire “wiring” in the HAL layer. Might keep it from becoming a multiprocessor nightmare, anyway…

  4. #7 by solaandjin on 2014-07-28 - 10:06

    I just took a quick look around youtube, and there are several videos with that screen and the Arduino. The refresh rates don’t seem as bad as you are reporting, but most of them don’t appear to be blitting bitmaps, and just using drawing primitives. Here’s a representative video: https://www.youtube.com/watch?v=7yzwKZgNqBg

    Perhaps an uglier but more responsive UI would suit your needs better.

    • #8 by Ed on 2014-07-30 - 14:24

      using drawing primitives

      Those are a bit peppier, but the secret is to draw as few pixels as possible, which requires keeping track of what changed, which seemed like a lot of work for a microcontroller with no RAM. I’d hoped to just overlay the button images and be done with it, but that obviously won’t suffice.

      Ugly is as ugly does: first, make it work, then make it pretty. I’m not yet at Step Zero!

  5. #9 by Neil H. on 2014-08-03 - 12:52

    Would something like a Raspberry Pi with an Arduino attached (e.g. a Pi Alamode IIRC) let you render the UI fast and still have the arduino motor control working (to re use the code you already have). Just thinking out loud here. Best wishes.

    • #10 by Ed on 2014-08-04 - 17:27

      That would work, but just the thought of setting up a multiprocessor system for a sewing machine makes my head hurt. It’s simply the wrong display for an Arduino, which makes it the wrong display for this application: now, I know.

      • #11 by Neil H. on 2014-08-04 - 17:38

        Agreed, the thought of an asymmetrical dual core control system for a sewing machine does seem to be overkill, and dealing with the multi-threaded :-) nature of the code between the 2 boards could be a hassle.

  6. #12 by Derek Lewis on 2014-08-08 - 11:44

    I was using the Adafruit 1.8″ display, and drawing images was too slow for my project, so I used imagemagic to convert the images on the SD card to the exact pixel format (RGB565 I think) that the screen needs (might be different than yours), without any header or anything, and then just read a chunk of pixels from the SD card, and send it to the screen. It sped it up about 3x.
    https://github.com/lewisd32/ArduinoSlideShow

    • #13 by Ed on 2014-08-08 - 12:20

      sped it up about 3x

      That would reduce the time from awful to tolerable, which is not to be sniffed at: 100 ms for the larger buttons seems Good Enough. I like it!

      The ILI9341-specific code in the Adafruit driver looks pretty much the same as for the 1.8 inch panel. Perhaps intense ImageMagick-fu could crush an RGB triplet into two 5-6-5 packed bytes: the SD card file would then hold the exact display data and the transfer loop wouldn’t need the mask-and-shift alignment step.

      Or live dangerously: have the Arduino read RGB triplets and write a raw file with the aligned display data! The first activation of each button would be glacially slow, but all the rest would be much faster, with the SD card acting as a cache. That makes no sense; better to do the conversion once (with ImageMagick or Python or whatever) and write-protect the SD card.

      Must. Stop. Complexicating.

      I still doubt there’s enough CPU for simultaneous motor control and screen updates, but predigesting the pixels will certainly help.

      Thanks for the suggestion and the examples!

      • #14 by lewisd32 on 2014-08-08 - 12:23

        Could the motor control be done in an interrupt, triggered on schedule by the watchdog timer?

        It would appear the example code in the repo I linked doesn’t do the rgb565 format output. I’m not sure where the code for that went, but I know you’re adept with imagemagic anyway. :)

        • #15 by Ed on 2014-08-08 - 13:25

          done in an interrupt

          That’s pretty much the only way to make it work, as you want very regular control inputs and outputs, but I must run the numbers before jumping to conclusions.

          The conversion from RGB to 5-6-5 happens in Adafruit_ILI9341.cpp (or similar for the 1.8 inch panel), where it produces the uint16_t consumed by pushColor. My “big” buttons measured 120×64 pixels, so they’d require just 240 bytes per row (the sample code buffered 20 RGB pixels = 60 bytes) and maybe maybe maybe there’s enough RAM for at least half a row.

          adept with imagemagic

          Or at least adept at finding examples suggesting how to do something similar, then fumbling from there… [sigh]