Newer
Older
# pitracker
This repository contains scripts to use the PiTracker.
[Hardware](#hardware)
The Raspberry Pis (rPis) can be set up prior building the actual eye-tracker, and in facts it might be easier.
1. Requirements:
- 2 Raspberry Pi zeros (https://www.raspberrypi.com/products/raspberry-pi-zero-w/). Ideally one with pins and one without.
- 2 SD cards. I've had good experience with the SanDisk Extreme 64GB.
- 2 rPi cameras. Cameras for the regular rPi and rPi zero are not compatible. For the scene-camera I prefer the spy camera with a 120deg angle and for the eye-camera the spy camera with a 15cm cable.
- 1 BNO055 accelerometer (https://www.adafruit.com/product/2472).
- 1 LiPo SHIM (https://www.adafruit.com/product/3196). Unfortunately I've had issues with this LiPo SHIM.
When 2 rPis are connected to the same battery, the current drawn through the SHIM appears too high and damages it.
If you are using only 1 rPi, then this should work.
But if you use 2 rPis then use a power bank and the rPis micro-USB port instead.
- 1 LiPo battery. The size depends on your application, but 2000mAh allows over 1h of stable recording (e.g. https://www.adafruit.com/product/2011).
- 1 mini-HDMI to HDMI-A adapter (https://www.raspberrypi.com/products/mini-hdmi-c-male-to-standard-hdmi-a-female-adapter/)
- 1 micro-USB to USB-A adapter (https://www.raspberrypi.com/products/micro-usb-male-to-usb-a-female-cable/)
- The NoIR cameras are pretty short. They can be extended using an FPC cable (e.g. https://www.mouser.de/ProductDetail/Molex/15166-0238?qs=N2VrfF4LzQeYxI9dzxiZfA%3D%3D). Different lengths exist and any cable would work as long as they have 22 pins, 0.5mm pitch and Opposite Side Contacts.
- If using an extension cable, you'll also need an FPC extension plug (https://www.berrybase.de/en/zero-kamera-kabel-joiner-fuer-raspberry-pi-22-pin-zu-22-pin).
2. Stack the Raspberry Pis: If you purchased one rPi with pins and one without, simply stack the pin-less rPi on the one with pins and solder every pin. When pressed as close as possible, the rPis will have a small angle. You can use spacers to keep them straight but I've had no issues without.
3. Add the LiPo shield: Stack the LiPo shield on top of the 2 rPis. The pins should be long enough to allow you to solder it as well.
5. Add the synchronization plug: By default the IR LED pin is set to GPIO pin 13 (pin 33 of the header, see pinout below).
The synchronization and trigger pins are set respectively to GPIO pins 20 and 21 (pins 38 and 40 of the header).
The state of these 2 pins will be recorded alongside the timestamps of video frames in a text file.
Warning: It is said that the rPis SD cards get corrupted over time. To preserve them try to boot off the rPi from the "Log Out" menu and not by unplugging power.
Similarly try to get a battery large enough that you can turn off the system at the end of the recording rather than the battery running out.
That being said, I have not yet experienced a SD card getting corrupted so this is not something to worry too much about.
For simplicity, a bash script can set up the rPis automatically. You still need to complete step 1 of manual installation below. Then to use the script, start by downloading this git repository, then call the bash script as root and provide the home folder as argument as follow:
```
cd ~/Documents
git clone https://gitlab.uni-marburg.de/caziot/pitracker
cd ~/Documents/pitracker
sudo sh setup_pitracker ~
```
This script will automatically perform steps 2, 4 and 6 below. Step 3 is necessary only if you use a local router.
The script will prompt you for a password for Samba at some point. Set any password you want (though it's probably easier to provide the same password as for raspbian), and at the end it'll ask you to reboot. After reboot, test that the camera is working with `raspistill -d`.
Note: necessary steps are indicated by numbers. All necessary steps need to be taken, not necessarily in order. When different solutions are available, they are denoted by dots. Only one of the possible solution denoted by a dot needs to be taken.
1. Install NOOB: Install NOOB on a SD card, start the Pi.
https://www.raspberrypi.org/documentation/installation/noobs.md
**Be careful to change the rPis default passwords**, automated scripts scan the internet and automatically log in rPis with unchanged default passwords.
Below I assume that you chose the username "pitracker". Change commands when necessary.
2. Activate the camera module. This steps depends on the version of NOOBS you are using.
1. On older versions of NOOBS:
- Using the graphical interface: go to Preference>Raspberry Pi Configuration>Interfaces and enable the camera module, then reboot the Pi. https://projects.raspberrypi.org/en/projects/getting-started-with-picamera/1
- Using the terminal: type ```sudo raspi-config```, select "3 Interface Options">"P1 Camera">Enable camera
2. On new versions of NOOBS, activating the camera is not available in the GUI. Instead, the "Legacy camera" needs to be activated using raspi-config. Using the terminal, type ```sudo raspi-config```, select "3 Interface Options">"I1 Legacy Camera ">Enable.
You can test the camera by typing ```raspistill -o ~/Desktop/test.jpg``` in the terminal. A JPG image should be created on the desktop. Or ```raspistill -d```, which should display the camera for few seconds.
3. Set up the wifi network. This step is necessary only if you plan on triggering the recordings from a computer and/or sharing data over wifi (software trigger). However it is possible to use the rPis in standalone mode only, trigger recordings with an electrical pulse (hardware trigger) and recover data offline by plugging the SD cards to another computer. Here I assume that you are using a wifi router to connect the data server to the raspberry Pis. Some institutions set a fixed IP address to every host based on their MAC address, in which case you must skip the first 4 steps.
1. Figure out the IP adress of the router. Typically it is 192.168.0.1 or 192.168.1.1.
2. On the rPis you can set static IP by editing the dhcp configuration file: ```sudo nano /etc/dhcpcd.conf```. Add the following lines:
```
interface wlan0
static ip_address = 192.168.0.101/24
static routers = 192.168.0.1
static domain_name_servers = 192.168.0.1
```
Here the rPi IP address will be 192.168.0.101, using the network mask 255.255.255.0 (/24). The router address is 192.168.0.1. The DNS server doesn't matter if you do not connect the rPis to the internet. Change these parameters accordingly to your configuration.
3. Quit nano with Ctrl+X, type "Y" to save, do not change the filename to overwrite it. Then restart the DHCP service with ```sudo /etc/init.d/dhcpcd restart```.
4. Set a fixed IP on the data server. Depending on the OS the computer you use to collect data:
- Windows: go to "Control Panel">"Network Connections". Right click on the wifi connection and select "Properties". Then select "Internet Protocol Version 4 (TCP/IPv4)" and click on "Properties". Select "Use the followinfg IP address" and set the IP address to 192.168.0.100, Subnet mask to 255.255.255.0 and gateaway to 192.168.0.1 (if that is the address of your router). On Windows the terminal command to display connections is ```ipconfig```.
- Mac: go to "System Preferences">"Network". Select Wifi and click on "Advanced". Here in the "TCP/IP" tab select "Configure IPv4: Manually" and set the IP address address like described above for Windows.
- Linux: the process is the same as for the Raspberry Pis. Edit the dhcpcd.conf file, set the wlan0 connection to the same fixed address described above then restart the DHCP service.
5. Before setting up a shared folder between computers, make sure the network works as expected by checking the IP address of each computer (```ipconfig``` on Windows and ```ifconfig``` on Linux and Mac, on Linux you might have install net-tools: ```sudo apt-get install net-tools```). Then make sure each computer can see other computers using ```ping 192.168.0.???``` (change ??? for the IP address of the computer you want to test). You should see "Reply from 192.168.0.???: bytes=32 time=?ms TTL=64".
4. Set up shared folders:
1. Install Samba: ```sudo apt-get install samba```. If you are asked something about DHCP what you select doesn't matter since we will use fixed-IP.
2. Create data folder ```mkdir ~/Data```. This folder can be anywhere on the disk, here it is created directly in the user's home. Change the rights to the data folder with ```chmod 777 ~/Data```, to allow everybody to read and write in that folder.
3. Edit the Samba config file: ```sudo nano /etc/samba/smb.conf```.
- Required: at the end of the file in the "Share definition" section add the following:
```
[PiShare]
comment=Shared data folder
browseable=Yes
writeable=Yes
only guest=no
create mask=0777
directory mask=0777
public=no
```
If you created a data folder at a different location, change the path line accordingly.
- Optional: I have not had to do this, but on some network it might be necessary to also edit the "Global" section as follow:
```
workgroup = WORKGROUP
wins support = yes
```
Most likely you only need to add the ```wins support = yes``` line after workgroup in the global section of the config file. The workgroup name can be anything respecting the Windows group names guidelines (e.g. PITRACKER), but that name has to match on the different computers/rPis (WORKGROUP is the default group name on Windows).
- Finally quit Nano typing CTRL+X, type Y to save changes, do not change filename to overwrite previous config file.
4. Add a user to the authorized samba users list. If you set access to public=yes, this step is not necessary. To add the default Pi user type ```sudo smbpasswd -a pitracker```. Then enter a network password when prompted (it can be anything, though the same password as the Pi account would probably avoid confusion).
5. Finally restart Samba with ```sudo /etc/init.d/smbd restart```
6. Now you should be able to see the shared folder on other computers:
- In Windows you might be able to see the rPi in Network tab of the File Explorer. In any case you can go to "This PC">"Add a network location">"Next">"Choose a custom network location". (If Windows asks anything about broadband connection just quit that window to go back to the "Add network location" interface.) Here enter the name of the "share". In the described example it would be \\192.168.0.101\PiShare\.
- In OSX open Finder and select "Go">"Connect to server". You must use the name of the folder, not the Share. In the example above it would be "smb://192.168.0.101/pi/Data" (change IP address accordingly). If it asks for credentials enter "pi" and the network password you set on the rPi. You should also be able to connect as guest if you set the "public" parameter to yes.
- In Linux also use the path of the folder: Go to "smb://192.168.0.101/pi/Data" (change IP address accordingly), enter "pi" (or whatever username you chose) as user and the password you set for Samba.
5. Clone this repository and install required libraries:
1. Download this repository on all rPi zeros (typically inside "~/Documents"): ```git clone https://gitlab.uni-marburg.de/caziot/pitracker.git```
2. Install libraries: ```pip3 install picamera pynput netifaces pysmb parse```
6. Set up VNC "Virtual Network Computing". Raspberry Pis come with a VNC server preinstalled. Next step you'll set the scripts to start on boot so you shouldn't have to use VNC during actual recordings. However it is very convenient to control the Pis during testing and debugging. You need to activate VNC on each rPi then install the VNC Viewer on your server to remotely connect to the rPis:
1. Activate VNC on the rPi: go to Preferences > Interfaces > Enable VNC (or use raspi-config).
2. Activate Direct Capture Mode: in VNC server, go to Options>Troubleshooting and activate Direct Capture Mode.
3. On your server, download VNC Viewer for your OS: https://www.realvnc.com/en/connect/download/viewer/
~~7. Set up python scripts to start automatically on boot. You probably want to do this at the last time, once you know everything is working as expected.
- To do this edit the file rc.local: ```sudo nano /etc/rc.local```. Add the following line before the ```exit 0``` tag: ```sudo -u pitracker /usr/bin/python3 /home/pitracker/Documents/pitracker/pitracker_scene.py &```. Of course this line should be added on the rPi connected to the scene camera and will work only it the git repository has been cloned into Documents. Change the path accordingly and change for ```pitracker_eye.py``` on the rPi connected to the eye camera.
- Here `sudo -u pi` is required because rc.local runs as root, therefore the python libraries installed as user pi will not be available. Another solution is tpo install all libraries using `sudo pip install ...`. Although as usual it is not recommended to work as root if not required.
- Similarly `/usr/bin/python3` is required since the PATH is not instantiated yet. The ampersand tells the OS to fork a thread to run the script.
- Make rc.local executable: `sudo chmod +x /etc/rc.local`
- Finally, you can test that everything is working fine using first `sudo systemctl status rc-local.service` to check if any error has been returned, as well as `cat /var/log/syslog | grep rc.local` to get a more detailed log of the errors.
- Do not forget to comment out the line in rc.local if you do not want to script to start automatically at startup. Remember that in the worst case you can always unplug the SD card and edit the /etc/rc.local file from a different computer.~~
7. Set up python scripts to start automatically on boot. You probably want to do this at the last time, once you know everything is working as expected.
- To do this copy the file pitracker_eye.service using: `sudo cp ~/Documents/pitracker/pitracker_eye.service /etc/systemd/system/pitracker_eye.service`.
- This is assuming that you cloned the repository in Documents. For the scene camera replace ```pitracker_eye.service``` by ```pitracker_scene.service```.
- Activate the service with `sudo systemctl enable pitracker_eye.service` (and _scene.service for the scene camera).
- The script should now start automatically on boot. You can check the status of the service with `systemctl status pitracker_eye.service`.
- If you change the service file, use `sudo systemctl daemon-reload` to update the service.
- You can deactivate the service with `sudo systemctl disable myscript.service`.
- In case of problems you can check the error log with: ```journalctl -u pitracker_eye.service```.
8. If you have activated the python script to start on boot, then there's no real need to run the X server on the rPi.
- To be able to easily reactivate the X server later, first turn on SSH in `sudo raspi-config` > 3 Interface Option > I2 SSH. You can now log in remotely in another computer's shell using `ssh pitracker@ipaddress` (replace pitracker by your username and ip address by the device's actual IP).
- Now turn off the x-server. Usually this would be done using `sudo update-rc.d lightdm disable`. But apparently this is not enough on raspbian, and the following command is also required: `sudo systemctl set-default multi-user.target`. Then `reboot`.
- To turn on the x-server again, log in through ssh and type `sudo service lightdm start`. You can now use VNC again. Or if using a monitor type ctrl+alt+F2, then the same command.
9. At this point you should have set up the rPis and be ready for recording. You can test the system by running the python script. Otherwise, when you'll reboot the rPi the python scripts will start automatically in background.
## Usage
The PiTrackers allow different ways to trigger recordings.
1. Trigger recordings:
- From the server (recommended): for convenience I provide a python script to run on a separate computer. This script will connect to the rPis and trigger recordings remotely. First modify the rPis IP addresses at the top of the script.
- Using VNC: connect to the rPis using VNCviewer. Enter the rPi IP address in the search bar (e.g. 192.168.0.101). When prompted enter the rPi login info ("pitracker" and the rPi password). You should now see the rPi desktop as if it were connected to a monitor. You can manually start the python script on each rPi. The console should display "Running...". You might see an error message about a channel already in use which is unimportant. Now you can preview the camera by pressing "p". If you are satisfied with the image you can trigger a recording by pressing "r". The rPi will automatically create a video file in the folder "~/Data/" named after the date and time of the recording. After pressing "r" a second time the recording will stop. Note that if you performed step 7 of the installation section the script will already be running in background so you should be able to trigger recordings by simply pressing "r" after having connected remotely.
- Using the recording pin. By default, this option is turned off to avoid starting or stopping a recording accidently. However, you can enable this option by setting ```allowRecordingPin = 1``` in the rPi global.ini file. In this case a recording will be triggered by sending a pulse to the pin 26 (the second pin at the bottom left).
2. Synchronize recordings: during recording, the rPi stores frame times in a .txt file alongside the video file. Pin 21 (the bottom right pin) is also monitored at each frame and its value stored in the same text file. Because the 2 rPis are soldered together the values of pin 21 can be used to synchronize the recording offline, as well as with separate recordings such as behavioral or neural data. However, be careful because the rPi are 3.3V devices. **Do not send 5V pulses** (such as from an arduino) to the rPis, they will burn right away.
3. Recover video files:
- Using the server script: When the server script triggers a recoding, it receives and stores the file name through the socket connection. You can then use the script to recover these files by pressing "f". File transfer will be very slow, do not interupt the script before the end. In any case the video files will remain stored on the rPis, so you can recover them even if something happens during file transfer. This also means that you must delete them manually to save space.
- Using a file explorer: connect to the samba shared folder as in step 4-6 of the installation section. You can manually copy/paste files to a different computer. Here again the file will remain on the rPi SD card until you manually delete them.
Allow I2C communication with the accelerometer:
1. Install libraries: ```sudo apt install i2c-tools python3-smbus```.
2. Activate I2C communication: ```sudo raspi-config```>"3 Interface Options">Enable I2C, or as usual go to options to do the same thing. Restart the rPi.
3. Get the address of the I2C slave: in terminal type ```i2cdetect -y 1```. The address is 7 bits in heaxadimal (thus has to be written 0xNN with NN the number returned by i2cdetect). The default address address should be 0x68.
4. Change the i2c baudrate to 50kHz:```sudo cat /sys/module/i2c-bcm2708/parameters/baudrate``` and ```sudo modprobe i2c-bcm2708 baudrate=50000```
5. Install the Adafruit BNO055 python library:
```
cd ~
git clone https://github.com/adafruit/Adafruit_Python_BNO055.git
cd Adafruit_Python_BNO055
sudo python3 setup.py install
```
If you get a syntax error, you are using python2 and not python3.~~