Displaying webcam video on Raspberry Pi using pygame

Categories Open-source, Programming, RMIT, Visuals

Another small exploratory project related to my undergraduate Major Project had me trying to stream video onto a small 3.5″ TFT LCD display using a webcam connected to a Raspberry Pi mini computer.

 

Firstly, not all USB webcams work with the Raspberry Pi, so after trying randomly to find a working webcam I discovered this list of Raspberry Pi verified peripherals and bought myself a Logitech C100 for $20 on ebay.

Secondly, USB webcams seem to be horribly slow on the Raspberry Pi, pulling dodgy framerates at what might be <10fps at their full resolution (in my case 640×480 pixels). Originally, I was using the custom imgproc library (available here) but found that there was either a lack of simple documentation for extended functionality like scaling and so on, or that the library wasn’t built to perform those sorts of tasks. Eventually I settled upon using the pygame library (which you can get here). The documentation for pygame is thorough and easy to navigate, making troubleshooting and extension or exploration of the library very easy to do.

In order to combat the low framerate and because resolution is not a priority (for the purposes of the concept I will be testing) I decided to pull the webcam feed in at a much lower resolution (32×24 pixels) and then scale it up to full screen (640×480 pixels). I’m unsure if the webcam can actually pull the feed in at 32×24 pixels, I have to look into that – and perhaps some pixel dropping techniques.

I wrote the code, copied it over, connected the webcam to the Raspberry Pi and ran the following python script:

import sys
import pygame
import pygame.camera

pygame.init()
pygame.camera.init()

#create fullscreen display 640x480
screen = pygame.display.set_mode((640,480),0)

#find, open and start low-res camera
cam_list = pygame.camera.list_cameras()
webcam = pygame.camera.Camera(cam_list[0],(32,24))
webcam.start()

while True:
    #grab image, scale and blit to screen
    imagen = webcam.get_image()
    imagen = pygame.transform.scale(imagen,(640,480))
    screen.blit(imagen,(0,0))

    #draw all updates to display
    pygame.display.update()

    # check for quit events
    for event in pygame.event.get():
        if event.type == pygame.QUIT:
        webcam.stop()
        pygame.quit()
        sys.exit()

After a brief pause a pygame window pops up and the webcam feed is shown on the screen, scaled to fullscreen. The framerate is better than the full resolution webcam feed, but is still very slow by modern standards.

 

Drawing in Python using Arduino data via serial

Categories Open-source, Programming, Visuals

Following on from my previous post where I established serial communication between an Arduino and a Raspberry Pi, I needed to find out if I could store the data being sent from the Arduino in a variable on the Raspberry Pi and use it to alter some functions in a Python script.

I wrote up a piece of code that tries to read the data from the serial and use it to change the colour of a circle drawn in the center of a 640 x 480 pixel screen:

# Title: Arduino2RPiSerial
 #
 # Goal: to draw a circle based on colour values sent from the arduino
 #

import pygame
 import sys
 import serial

pygame.init()

#create a screen
 screen = pygame.display.set_mode((640,480))

#create colour
 redValue = 255
 fgColour = (redValue,0,0)
 bgColour = (0,0,255)

screen.fill(bgColour)

#serial(device,baud)
 ser = serial.Serial('/dev/ttyACM0',9600)

# LOOP
 while 1 :

#Update from serial
 redValue = ser.readline()
 #update colour value
 fgColour = (redValue,0,0)
 pygame.draw.circle(screen, fgColour, (320,240), 200, 0)
 pygame.display.update()
 pygame.time.delay(5)

# check for quit events
 for event in pygame.event.get():
 if event.type == pygame.QUIT:
 pygame.quit(); sys.exit();

Getting the Raspberry Pi to talk to Arduino

Categories Open-source, Programming

As part of an exploratory design project I am undertaking within my undergraduate Major Project, I needed to establish serial communication between a Raspberry Pi (a miniature, credit-card sized computer) and an Arduino Uno (an electronic prototyping board). I found that Raspberry Pi could both supply power and talk to the Arduino Uno via USB cable.

I also found that as a beginner in many aspects of Python programming, Linux commands and serial communication that much of the documentation on the internet seemed daunting, so I’ve made a very thorough documentation here to help anyone out who has the same amount or less of the starting knowledge that I had. Hopefully it can aid in the creation of more open-source Raspberry Pi / Arduino projects.

I based my efforts on the steps outlined here.

 

 

 

Arduino Uno, Raspberry Pi and Laptop.
Arduino Uno connected to the laptop, with  a Raspberry Pi bundled with it’s own 3.5″ LCD Screen running below.

 

Firstly, I needed to install pyserial onto my Raspberry Pi. I have no ethernet cable or Wi-Fi dongle connected to my Raspberry Pi and so I had to transfer the pyserial-2.6.tar file via a USB Flash drive. The file can be downloaded here.

After transferring the file to desktop (home/pi/desktop) and unzipping it’s contents to it’s own folder on the desktop (home/pi/desktop/pyserial-2.6) I opened up the LXterminal to execute the following commands:

cd /home/pi/desktop/pyserial-2.6

The cd command directs the terminal to a folder on the Raspberry Pi (in this case, the desktop), to give future commands a context of operation. Basically, the command tells the terminal where we are working from and where to look for any files called upon.

Next:

sudo python setup.py install

This command tells python to install pyserial. I hit enter and the terminal spat out line after line of output, completing the install process shortfly after.

From this point, the Raspberry Pi was ready for serial communication via the pyserial module.

I needed to find out what the device name for the Arduino Uno was on the Raspberry Pi so that I could reference it in the python shell when opening up serial communications. For this I used a command that lists the devices:

ls /dev/tty*

The ls command lists files within a directory, the second part ‘/dev/tty*’ is a filter. The use of the wildcard character ‘*’ means that we are looking for any files that begin with ‘/dev/tty’ and that end with any other combination of characters. I ran the command without the Arduino Uno connected and then again with the Arduino Uno connected so that I could see the difference between the two lists, and find out what it’s device name was.

 

 

 

Using ls /dev/tty* to display a list of devices
Using ls /dev/tty* to display a list of devices

 

It’s hard to see in the picture above because the 3.5″ TFT LCD screen has such a horrible resolution (and has a strange flickering colour issue) but in the second list positioned at 3 items from the bottom of the rightmost column, a file appeared that wasn’t present in the first list (/dev/ttyACM0). I made note of that file because I would need to reference it later.

Then, the code below needed to be uploaded to the Arduino Uno to generate a test signal:

//Code originally by simon monk (http://www.doctormonk.com/2012/04/raspberry-pi-and-arduino.html)
//Commented by Daniel Kerris

//This code makes use of the Arduino's built-in LED which is located on Pin 13
const int ledPin = 13;

 //This sets up the code and defines the baud rate as 9600 - it's important to use the same baud rate on the Raspberry Pi
 void setup()
 {
   pinMode(ledPin, OUTPUT);
   Serial.begin(9600);
 }

 //This part of the code loops continuously with a delay of 1000 milliseconds between each loop
 //It sends the message "Hello Pi" over and over again via USB
 void loop()
 {
   Serial.println("Hello Pi");

   //This section checks to see if the Raspberry Pi is talking back
   //and then makes a call to the flash() script which is define below
   if (Serial.available())
   {
      flash(Serial.read() - '0');
   }
   delay(1000);
 }

 //This flash() script takes the number that the Raspberry Pi has sent over via USB (stored in the variable n)
 //and uses it to determing how many times the LED should flash
 void flash(int n)
 {
   for (int i = 0; i < n; i++)
   {
     digitalWrite(ledPin, HIGH);
     delay(100);
     digitalWrite(ledPin, LOW);
     delay(100);
   }
 }

After the code was uploaded to the Arduino Uno, I connected it via USB to the Raspberry Pi. I could see that it was powered and that the code was working because the LED for Tx (serial communication) was flashing, showing that it was being used. The Raspberry Pi still needed some code to tell it to listen to the messages coming through the USB port.

I booted up the Raspberry Pi and opened up the application called IDLE, highlighted the Python Shell Window and entered the following commands, hitting enter after each line:

import serial
ser = serial.Serial('/dev/ttyACM0', 9600)

You’ll remember that ‘/dev/ttyACM0’ was the name of the Arduino that I had found out earlier. 9600 is the baud rate, which is set the same is in the Arduino code above.

Then:

while 1 :
     ser.readline()

The while statment is used to do a set of actions while a statement is true. In computer languages 1 and 0 are stand-ins for true and false or on and off. Basically, this line of code is saying that whilst 1 is equal to true (which is constantly) then do something. This creates a loop that executes itself until the program quits, just like the void loop() in the above Arduino code. The ser.readline() is telling our serial object ser that we want to execute the function readline() which reads from the serial device (‘/dev/ttyACM0′) that we established earlier.

I had to hit enter twice after the ser.readline() command to let Python know that I am finished writing commands within the while statement’s block (you can read more about code blocks here if you don’t understand).

 

 

Success!
Success!

 

Finally, the Raspberry Pi successfully read from the Arduino.

I couldn’t get return communication happening successfully, though luckily for me the project I am working on only requires one-way communication from Arduino Uno to Raspberry Pi and not the other way round. I’m going to be using the Arduino Uno to handle sensors and tangible controls as I am much more comfortable coding and prototyping in that environment. I will write code that analyses and packages the information from sensors and tangible controls and sends it via USB to the Raspberry Pi, which will handle all of the GUI and Sound output (which the Arduino would otherwise struggle to handle).