Understand Arduino development

Arduino is a fantastic platform for getting started with embedded software development. You are provided a development board, and programming IDE all configured to work together immediately. As a developer, all you need to do is write your program, press a button, and it is compiled and uploaded to the board where it begins execution.

It's important you understand the fundamentals of what is happening in the background to make this work for you. It's especially important when you're trying to debug cryptic errors that would make sense in a conventional development environment, but not the Arduino IDE.

The language

Lets start with the first statement I hear often:

"Arduino is running some bastardized version of C"

Or better,

"I think Arduino uses some language called Processing or Wiring or something"

The Arduino IDE compiles your project as C++ using the GNU AVR toolchain. A toolchain refers to the set of programs that take your source code and transform it into the binary instructions the processor executes directly. GNU AVR is a free and open-source project on its own, that Arduino uses.

So you are writing "real" C++ code, though there are a few hardware limitations to keep in mind. It's called the GNU AVR toolchain because it compiles code for the AVR micro-controller architecture. You can check up more at the projects homepages:

Processing and Wiring are other projects Arduino is related to. Processing is an entirely other language and IDE that are tightly coupled together. The IDE itself is written in Java, and is used by the Arduino project as the code editor and interface. None of the actual processing language is used.

Wiring is a project similar to Arduino; it has a development board, processing based IDE and supporting libraries. Arduino uses a Wiring library for some functions, like the digitalWrite and digitalRead.

Given the limitations of the hardware, some common (practically standard) libraries such as C++ STL are not available by default, however members of the community have made these available as well.

Arduino IDE

Source file mangling

UPDATE: Although there is some source file processing before compilation, I have been informed that it is not be copying and pasting the contents of the main sketch into the main.cpp file. I'll post more on this when I can.

To see the generated cpp file, you can look at the temporary build directory by turning on verbose compilation settings and going to the directory listed.

UPDATE (Aug 4 2014): Did a little reading of the Arduino build process here: http://arduino.cc/en/Hacking/BuildProcess, look under transformations to the main sketch

.ino files are actually C++ files, the difference being that they are mangled by the IDE before being compiled. INO (.ino) files are contained within a folder of the same name and the whole thing is called a "sketch".

- mysketch/
    -> mysketch.ino
    -> my_other_source_file.ino

When you compile your sketch, the IDE cuts the setup and loop functions and puts them into a temporary C++ source file with a structure like: the IDE copies your sketch into a temporary directory and manipulates the sketch by:

  • Prepending #include statements for the standard Arduino libraries
  • Generating prototype definitions and placing them just after the pre-processor statements
  • Appending the following code to the end of the file
int main( int args, char* argv[] ) {
    setup();

    for(;;)
        loop();
}

There is a little more to it than this (some manipulation of global variables and header information), however this is the core of whats happening. Setup is called once, and the loop function is called forever.

If there is more than one .ino file, it is simply appended on to the end of the main .ino file.

Ever wonder what happens if you were to break out of the infinite for loop? Well there is no operating system on this little chip, so there is nothing for the program to hand control over to when execution completes. Usually the board just resets itself if this ever happens.

Uploading code

After the code has been compiled to a temporary directory, its passed to another open-source program called avrdude, which has been configured to upload to whatever board is selected in the Arduino IDE.

When you connect the Arduino to your computer, it is recognized as a serial device. This is because there is a chip between your computer and the actual micro-controller that allows you to communicate to it using a serial protocol. On older board revisions this was standard FTDI USB-Serial chip, but was replaced with a micro-controller unit later. So your Arduino board likely has two micro-controller units! The difference being that the interface micro-controller is tiny and cannot be programmed easily by the user and has one specific purpose; emulating a serial port over USB.

Avrdude sends a specific byte sequence that tells the main micro-controller to restart and switch over to its bootloader. The bootloader is a program pre-loaded in a reserved area of program memory whose purpose is to overwrite the normal program memory with your program. After the bootloader is finished, the device restarts and starts execution of the newly loaded program.

It's possible to overwrite the bootloader with a different one, or remove it completely. If this happens, you can no longer program the micro-controller over the USB cable. Its still possible to program it, but you require a standard AVR programmer. If you look at the pins available on the Arduino board, you will notice two six pin sets, one at the bottom centre, and another near the top right (assuming top to be where the USB header is facing).

The set of pins on the bottom are for connecting a standard programmer to the main micro-controller, and the top set are for programming the USB-Serial micro-controller.

Techniques for large projects

The Arduino IDE has the concept of a "sketchbook", and the programs that you write are called "sketches". So the sketchbook is a folder which contains all of your sketches. I previously mentioned briefly that a sketch is a folder that contains a file with the same name as the folder itself, and possibly any additional source code files. The sketchbook should also contain a folder called "libraries", which allows you to share code across sketches.

Write libraries

Similar to structure of a sketch, a library needs to be a folder where the name of the folder is the name of the library. However, you don't need to make a source file have the same name as the folder.

When you write a library, the process is a little different than writing a sketch, and there is no source file mangling. Arduino has a [tutorial page for writing libraries][arduiino_0] that breaks down the process. Note that you don't place an .ino file in the library.

The main reason you would consider this is because you should split your project into multiple files by their logical function. You are not limited to using libraries to achieve this either, a sketch can have multiple source files. The source files don't need to be .ino either, only one is required to make the sketch valid, you can also use .c, .h and .cpp, although these require you to include a few things that would normally have been done for you.

Splitting the project into modular files is key for long term development. It allows you to reuse code later on without having to copy and paste individual functions out of source files, rather you just include the library or add the source files to your sketch. When you make your project modular, you can document each module on its own, which allows others to use that module in their own projects without having to understand how the entire program works.

For the Arduino IDE to recognize new libraries, you must restart the program. The same applies if you want to change the sketchbook folder.

There are other reasons for doing this, however I want to touch base on how to use files that don't have the .ino file extension.

Using non .ino files

Header files end in .h and allow you to define data structures and classes. If you are having an issue where your structure doesn't work at the top of your .ino file, then try creating a separate .h file and include it from the main sketch, like:

#include "my_custom_header.h"

void setup() {}
void loop() {}

Where your sketch folder would look like:

-/sketch
    - sketch.ino
    - my_custom_header.h

Note the use of quotes (") rather than angle brackets (<). This specifies that the file to be included is in the same folder as the source file. When you include a library you should use angle brackets. What makes this more fun is that if you do use quotes to include a library it will still work do to Arduino's compilation process, although its bad practice.

So when you include a library it will look like:

#include <my_library.h>

void setup() {}
void loop() {}

And the sketchbook structure should look like:

sketchbook/
    - sketch/
        - sketch.ino
    - library/
        - my_library/
            - my_library.h
            - my_library.cpp

When you use the .cpp file extension, the Arduino standard library of functions is not available by default. You need to specify that you want to use standard functions like digitalWrite(int, int) you need to make sure you have included:

#include <Arduino.h>

If you wish to write code in C, you need to add some pre-processor tags to inform the C++ compiler that the functions are available. So for a library module with a header and source:

library_name/
    - my_library.h
    - my_library.c

my_library.h:

#ifdef __cplusplus
extern "C" {
#endif

// Actual code content

#ifdef __cplusplus
}
#endif

Just note that your c file will not be able to call the standard Arduino functions since they are written in C++.

Serial ports

If you write a library that uses a serial port, you should not use the hard-coded names of the serial ports, i.e. "Serial1", "Serial2", etc. Instead make a pointer to a Serial port type object. The default serial ports are pre-instantiated instances of type "HardwareSerial" class, which is a child of the "Stream" class.

So to maintain maximum compatibility, make your library use a pointer to a Stream type class. This way if a user decides to use a SoftwareSerial class, it should still work. This way you can pass a serial port to a function like

int my_function(Stream* inport) {
    inport->println("This is a test");
}

And call it like:

my_function(&Serial1);

Memory management

The Arduino has three different types of memory you need to keep in mind when developing your project:

  • SRAM (smallest)
  • EEPROM
  • Flash Memory (biggest)

SRAM is where all your variables and object instances are stored, it is equivalent to the RAM on your desktop pc. You need to pay most attention to this, since its used the most during run-time and running out can crash your program.

Flash memory is where the program is stored, where all the instructions you wrote are retrieved and then executed. You have more of this than any other type of memory, however it is essentially read-only during program execution.

There is an Arduino tutorial page here:

http://playground.arduino.cc/Code/AvailableMemory

Which provides some methods of checking the amount of SRAM available.

Strings eat your ram! Due to the AVR architecture, when you declare constant strings in your code, the string is copied into ram at program execution and they stay there. The reason for this stems from the fact that AVR is a Harvard CPU archetecture.

You can get around this with some specific functions that keep the strings in flash memory until they are needed. The simplest way to do this to use a provided macro when you define strings:

Serial.println( F("This won't eat sram") );

Look up the details on AVR PROGMEM

MALLOC and NEW are dangerous but often necessary. They use SRAM and won't give it back unless you specify it explicitly, furthermore, you need to try and not fragment the memory.

When you declare memory off the heap with these functions, the memory won't be returned until you explicitly give it back. However even after giving the memory back, if there was more memory used after that location, there is now a hole of memory available. If the next malloc call doesn't need that exact amount of memory, then you end up with small wholes in the memory heap.

See this question on Stack Overflow for more information:

https://stackoverflow.com/questions/1334055/what-happens-when-stack-and-heap-collide

Segmentation faults

If you have been programming in C/C++ before, you might be familiar with the term segmentation fault. This is where you try to access memory that shouldn't be in the current context. Actions like accessing an array at an index that's too long. When this happens on desktop computer, the operating system will catch the error and terminate the program, or take some other corrective action.

Since there is no operating system on the Arduino, there is nothing to handle the error. So you end up with undefined behaviour, typically resulting in the device restarting.

This can make debugging difficult if you don't realise whats happening, since there will be no explicit notification.

Comments !

blogroll