Brain-Controlled Wheelchair An easy guide from
The parts of this system include an electric wheelchair, a laptop computer, an Arduino, an interface circuit, an EEG headset, and a collection of ready-made and custom software.
The software which was written specifically for this project (including the GUI and Arduino sketch) has been bundled with Puzzlebox Brainstorms, and is released freely under an Open Source license.
The EEG headset, which connects wirelessly to the laptop, allows the operator to simply think "forward" or "left" or "right" to cause the wheelchair to move. Performance is related to practice by the user, proper configuration of the software, and good contact made by the EEG electrodes on the scalp of the operator.
The interface circuit connects between the Arduino's digital pins and the joystick of the wheelchair. When the Arduino receives a command from the computer, it causes the circuit to "fool" the wheelchair into thinking that the operator has moved the joystick.
WARNING: Electric wheelchairs are designed for trained operators who have practised manoeuvring safely. Electric wheelchairs can weigh upwards of 200 pounds EMPTY! They have no mechanism to stop when they hit a human or an animal, or go off a curb or down stairs or right through a bannister. Make sure no one is in the path of a wheelchair which is not being operated by a human who can operate the OFF switch. Make sure everyone in the vicinity of this Instructable realizes what may happen if they don't get out of the way.
Do not attempt to stop an electric wheelchair with your body.
The software which was written specifically for this project (including the GUI and Arduino sketch) has been bundled with Puzzlebox Brainstorms, and is released freely under an Open Source license.
The EEG headset, which connects wirelessly to the laptop, allows the operator to simply think "forward" or "left" or "right" to cause the wheelchair to move. Performance is related to practice by the user, proper configuration of the software, and good contact made by the EEG electrodes on the scalp of the operator.
The interface circuit connects between the Arduino's digital pins and the joystick of the wheelchair. When the Arduino receives a command from the computer, it causes the circuit to "fool" the wheelchair into thinking that the operator has moved the joystick.
WARNING: Electric wheelchairs are designed for trained operators who have practised manoeuvring safely. Electric wheelchairs can weigh upwards of 200 pounds EMPTY! They have no mechanism to stop when they hit a human or an animal, or go off a curb or down stairs or right through a bannister. Make sure no one is in the path of a wheelchair which is not being operated by a human who can operate the OFF switch. Make sure everyone in the vicinity of this Instructable realizes what may happen if they don't get out of the way.
Do not attempt to stop an electric wheelchair with your body.
Step 1Acquire materials
an EEG Headset, (We used an Emotiv wireless headset
Laptop Computer, an Arduinoand an interface circuit
The EEG Headset is an EPOC acquired from Emotiv. Their site is here:
http://www.emotiv.com/
Step 2Software
The Software necessary includes the Puzzlebox Brainstorms BCI (Brain-Computer Interface) package which provides a GUI and visual feedback to the user and issues control commands to the Arduino hardware. Also required is the standard drivers and software included with the Emotiv EEG headset as well as an Aurduino Sketch which is a simple program written by us for an Arduino.
An optional Python-based backend control script can operate independently of the Puzzlebox Brainstorms GUI (see Step 7) and will translate keypresses entered on the keyboard into commands sent to the Arduino, which is connected to the interface circuit. The interface circuit makes the wheelchair move by simulating someone pushing on the joystick. This option is useful for controlling the wheelchair remotely via SSH session from another computer or device such as a mobile phone. Thebackend control script can be found here:
http://brainstorms.puzzlebox.info/tracker/browser/trunk/Puzzlebox/Brainstorms/Wheelchair_Control.py
And a suitable Python interpreter can be found here:http://www.python.org/download
The Puzzlebox Brainstorms software runs alongside the Emotiv headset's Control Panel andEmoKey application, which allows the wearer of the headset to cause characters to be "typed" on the laptop when a learning algorithm matches the user's current brainwave patterns to previously trained sequences.These "typed" characters activate buttons displayed in the Puzzlebox Brainstorms GUI (or the backend control script) which tells the Arduino to activate the Interface Circuit which simulates someone pushing on the joystick, which causes the wheelchair to move. The Puzzlebox Brainstormssoftware can be found here:
http://brainstorms.puzzlebox.info/static.php?page=downloads
There are two programs from Emotiv which we will use for this project: Emotiv Control Panel and EmoKey. Both are available to download freely as part of their "SDKLite" software pack:http://www.emotiv.com/store/sdk/edition/sdklite/
The Arduino Sketch is a program which is loaded into the Arduino hardware using free software from the Arduino site. The program watches for a series of characters coming in the USB serial port from the computer. When it sees the letter "x" it watches the next eight characters that come through, and they correspond to eight pins on the Arduino. If a character is "1" then that pin is set to ON, and outputs 5 volts, otherwise the pin is turned OFF and outputs zero volts.
The Arduino sketch can be found here:
http://brainstorms.puzzlebox.info/tracker/browser/trunk/arduino/puzzlebox_brainstorms_wheelchair_noisebridge/puzzlebox_brainstorms_wheelchair_noisebridge.pde
To load the above program into the Arduino, you need the Arduino software, found here:
http://arduino.googlecode.com/files/arduino-0021.zip
You might also need drivers for the USB serial interface on the Arduino, depending on your operating system and which version of the Arduino you own. Details at:http://arduino.cc
Step 3Test the wheelchair
Step 4Modify the wheelchair's joystick
To control the wheelchair electronically, we need to make a connection to its joystick. The joystick is a pair of variable resistors, one for forward/backward and another for left/right. Each variable resistor is connected between two voltages; one approximately zero and the other around 12 volts.Step 5Build the Interface Board
The interface board takes eight control lines in, from an Arduino or a PC parallel port, and can make the wheelchair think the joystick is being moved. It does this using transistors, some diodes and a bunch of resistors. All of these parts can be found at mouser.com or digikey.com or several other suppliers. Since the transistors (2N7000) are FETs, they draw no current (although their default-mode resistors take a milliamp or so) so they can be driven from any digital control signal. The diodes we used are 1N4148.
A datasheet for the 2N7000 transistor can be found here:
http://www.fairchildsemi.com/ds/2N/2N7000.pdf
The circuit contains eight transistors. Two of them pull the forward/backward wire toward ground through a resistor each. One has a larger resistor and the other a smaller one. When both transistors are turned on, both resistors are pulling toward ground, representing maximum speed in that direction. Another two transistors are used the same way for the left/right wire.
The other four transistors are connected between ground and a resistor to the 12 volt power supply. These transistors are normally on, and their drain terminal is connected through a diode to the forward/backward or left/right wire. When these transistors are turned off, the resistor is no longer pulled toward ground and the joystick wire gets pulled toward the +12v wire through the diode.
This is an easy way to let an eight-bit TTL (0 / 5V diigtal) controller like the PC parallel port or 8 pins of an Arduino control the 12-volt analog wheelchair joystick. In actual practice (as you can see by looking at the schematic and the actual circuit) the resistor values have to be chosen to achieve values which will not cause the wheelchair brain to think there is a malfunction in the joystick. (the 41K and 6.8K resistors are to pull the transistors' inputs to their default state, so the joystick works normally when nothing is plugged into the interface boards' input side).
A datasheet for the 2N7000 transistor can be found here:
http://www.fairchildsemi.com/ds/2N/2N7000.pdf
The circuit contains eight transistors. Two of them pull the forward/backward wire toward ground through a resistor each. One has a larger resistor and the other a smaller one. When both transistors are turned on, both resistors are pulling toward ground, representing maximum speed in that direction. Another two transistors are used the same way for the left/right wire.
The other four transistors are connected between ground and a resistor to the 12 volt power supply. These transistors are normally on, and their drain terminal is connected through a diode to the forward/backward or left/right wire. When these transistors are turned off, the resistor is no longer pulled toward ground and the joystick wire gets pulled toward the +12v wire through the diode.
This is an easy way to let an eight-bit TTL (0 / 5V diigtal) controller like the PC parallel port or 8 pins of an Arduino control the 12-volt analog wheelchair joystick. In actual practice (as you can see by looking at the schematic and the actual circuit) the resistor values have to be chosen to achieve values which will not cause the wheelchair brain to think there is a malfunction in the joystick. (the 41K and 6.8K resistors are to pull the transistors' inputs to their default state, so the joystick works normally when nothing is plugged into the interface boards' input side).
Perfboard on WikiPedia:
http://en.wikipedia.org/wiki/Perfboard
http://en.wikipedia.org/wiki/Perfboard
Step 6Program and connect the Arduino
You will need to download and install the Arduino software onto your computer. This is the program which will let you load the Sketch onto your Arduino which will make it perform its function in this system. The Arduino software communicates to the Arduino through a serial port which appears to your computer after the USB interface is plugged in.
If the Arduino is plugged in and your computer can't figure out that it is a serial port, you will need to download a driver from the Arduino website.
You will also need the sketch we have written for this application.
All of the above software is linked in Step Two of this instructable.
Once you have the Arduino software running, and it recognizes your Arduino hardware (which is plugged into your computers' USB port) you will need to load the sketch we have written using the File menu. Then go to the Tools menu and select "Board" and choose the model of Arduino you have. The Diecimila, Duemilanove or Uno are very similar and any of them would work for this application. Next, from the Tools menu, make sure the Serial Port of your Arduino is selected.
At this point, with our sketch loaded, you can go to the File menu and select "Upload to I/O board" to load the sketch into your Arduino. After this step works, you can quit the Arduino program. You won't need it again unless you want to change the way your Arduino operates, and it shouldn't be running while the Python program is running.
After this, you will need to make a connector to go between the Arduino and the interface board which controls the joystick of the wheelchair. We used pins 10, 2, 3, 4, 5, 6, 7, 8, of the Arduino and connected them to pins 2, 3, 4, 5, 6, 7, 8 and 9 of the DB25F connector from the interface board. We also connected Ground from the Arduino to pin 18 of the DB25F connector.
If it is not obvious at this point, you can skip the whole DB25 connector and just wire directly from the Arduino to the transistors on the interface board. Our setup is the result of originally using the parallel port of the PC instead of an Arduino (which is still an option if you have a parallel port and don't want to use an Arduino)
Step 7Install the Python interpreter and program
NOTE: The Puzzlebox Brainstorms software includes the console-based Python program described in this step. Strictly speaking it should not be necessary to download the Python interpreter or any of the software's dependencies (the Windows version mentioned in Step 8 is pre-compiled and packaged), but it is assumed anyone who wishes to leverage this software for a new type of wheelchair will need to know how to control everything from a source code level, so we have included these details for completeness.
Download and install the Python interpreter and download the Python program, listed in Step Two of this Instructable.
You may have some trouble configuring the program for the correct serial port of your Arduino, especially if your computer already has a serial port.
Make sure the Arduino is plugged in and run the program using the Python interpreter. If you press direction keys such as i j k m, you will see the built-in LED on the Arduino turn on or off because it is wired (in software) to one of the output lines to the interface board.
At this point you can connect the Arduino to the interface board. Prop the wheelchair up so its wheels are not touching the ground, or if you can (and we did) disengage the wheel clutches. This way, when the motors turn, they don't actually cause the wheelchair to move (although it no longer is braked in place and could roll).
If you turn on the wheelchair, the lights should come on normally and indicate that the wheelchair is ready to be moved. Moving the joystick should cause the brakes to click off and the motors to try to move the chair.
Step 8Install the Puzzlebox and Emotiv software
Puzzlebox Brainstorms is an open source BCI (Brain-Computer Interface) software package which allows control of various robots and vehicles via EEG. Version 0.4.5 of the software was updated to control the Action Arrow Storm Series wheelchair featured in this Instructable.
The software has been pre-packaged for Linux and Windows and is available for download at the website: (this was listed in Step 2)
http://brainstorms.puzzlebox.info/static.php?page=downloads
The software should be entirely compatible with Mac OS X, however it has not been packaged or tested with that operating system.
There are two programs from Emotiv which we will use for this project:
Emotiv Control Panel
and
EmoKey
Both are available to download freely as part of their "SDKLite" software pack:
(this was listed in Step 2)
Step 9Train the software and the human operator
To drive and control the wheelchair via EEG we will use the Emotiv Control Panel application to perform detections of the user's intent, turn those detections into keystrokes using EmoKey, and use those keystrokes to actually drive the wheelchair using Puzzlebox Brainstorms.
To begin, load the Emotiv Control Panel program and select or enter a user name for your user's profile. We'll be using the "Cognitiv" function of this software which is effectively a learning algorithm. The user will need to train the software to recognize their "Neutral" state - that is the user should sit still, and look away from the screen, clear their mind and try not to think about anything in particular. Then the user should select a direction to move a floating box on the screen, for example they can "Push" the box away from themselves or "Lift" it up in the air. Once trained, the software will attempt to detect the difference which occurs in the user's brainwaves between the neutral state and the patterns which occur when the user thinks about moving the box.
In our experience the Emotiv software is very good at training at least one direction direction for control, for example thinking "Push" to drive the wheelchair forward, and allowing that user to "turn on" or "turn off" that single command at will. However it seems that most if not all users have trouble training a second or third direction as effectively as the first. For this reason it is recommended when first using the software to "cheat" for left and right control by using facial muscles (which is technically EMG, not EEG) for example closing one eye to move left or right or clenching either side of the jaw.
Next we will use the EmoKey software to translate the detections made by the Emotiv Control Panelinto keystrokes which send to Puzzlebox Brainstorms which in turn controls the wheelchair.
An example file is available which uses "Push" to drive the wheelchair forward whenever the meter rises above 30% and issues a "Stop" command whenever the meter drops back below that level:
http://brainstorms.puzzlebox.info/tracker/browser/tags/brainstorms/release-0.4.5/emokey/puzzlebox_brainstorms-wheelchair.ekm
