Open Programmer v0.7.x

Open Programmer v0.7.x for pic

 Hi this is  an open source USB programmer for PIC micros, I2C and MicroWire EEPROMs, some ATMEL micros, generic I2C/SPI devices and (soon) other devices

Un programmatore USB open source per PIC, memorie EEPROM I2C e MicroWire, alcuni micro ATMEL, dispositivi I2C e SPI, e (in futuro) altri dispositivi

uick facts

• Completely free and Open Source (including firmware)
• Programs PIC10-12-16-18-24, dsPIC30-33, EEPROMs type 24xxxx (I2C), 25xxx (SPI), 93xx6 (MicroWire), some ATMEL micros, communicates with generic I2C & SPI devices (see supported devices)
• Can work as ICD debugger
• USB 2.0 Full Speed interface, HID class (same as keyboards, mice, etc.)
• Self powered
• Doesn't need drivers
• Built from easy to find components (estimated cost ~10€)
• Hardware generated timings for maximum speed and reliability (writes a 18F2550 in 15s, 8s under Linux)
• Doesn't saturate your CPU and doesn't suffer when other programs are running
• Open source control programs for Linux and Windows

USB & HID firmware (v0.7.6)

In order to use the USB interface included in some PIC devices we need a firmware that implements one of the classes defined by the USB consortium or a new one; I opted for the HID class, which is supported natively by all operating systems and so doesn't need any driver. Maximum allowed speed is 64KB/s, although with my application I measured something in the range 20-40 KB/s, certainly enough to program devices with memory of 100KB at most.
Like all USB devices this one too has a vid and a pid; these are usually obtained under payment, but since I don't have any money to waste and I'm not selling a commercial product I used the default Microchip vid and a pid of choice: 0x4D8&0x100; anyways it's possible to configure both, so I leave the choice to the user.
The programmer appears to the system as a HID device that exchanges 64 bytes packets every 1 ms.
The USB firmware comes from a nearly unknown open source project, written by Alexander Enzmann, which I modified and adapted to the MCC18 compiler.
I wrote a brief guide on how to use it; to my knowledge this is the only open source firmware with HID support and GPL license.
My programmer code simply adds a command interpreter that drives the microcontroller's outputs according to a set of instructions.
The main control cycle waits for a packet from USB, then executes commands in sequence while managing communication tasks; at the same time a control function is called periodically by a timer interrupt and keeps the DCDC regulator output voltage constant.
Building the project requires only free programs: MPLAB and MCC18 student version, which are unfortunately only available for the windows (in)operating system.
It's certainly possible to compile with SDCC, but some changes are needed to the source code.
Everything is given under the GPL2 license.
Here is the complete MPLAB project, here the hex file only.
Here a version for 18F2450 (with reduced functionality, see the circuit).

Command-line utility (Linux and Windows, v0.7.9)

OP is a command-line executable; the Windows version doesn't need installation;
the Linux version searches the programmer between devices /dev/usb/hiddevX (or the one specified as parameter) and needs reading rights for it.
eg. >sudo chmod a+r /dev/usb/hiddev0
To permanently enable a user do the following (on Ubuntu and other Debian based distributions, check for others):
as root create a file /etc/udev/rules.d/99-hiddev.rules
if you want to enable a user group write:
   KERNEL=="hiddev[0-9]", SUBSYSTEM=="usb", SYSFS{idProduct}=="0100", SYSFS{idVendor}=="04d8", GROUP="<group>"
where <group> is one of the user groups (to get a list type "groups"); select a suitable group and if your user desn't belong to it execute "addgroup <user> <group>"
or, if you want to enable all users, change reading permissions:
   KERNEL=="hiddev[0-9]", SUBSYSTEM=="usb", SYSFS{idProduct}=="0100", SYSFS{idVendor}=="04d8", MODE="0664"
restart udev to apply changes:
> /etc/init.d/udev reload

If after plugging the device /dev/usb/hiddevX is inexistent (and LED2 doesn't blink at 1 Hz), it's sufficient to execute a few times lsusb to force enumeration, or unplug and replug the cable.

If not otherwise specified the program looks for an USB device with vid&pid=0x4d8:0x100.
Reads and writes hex8 and hex32 files, and also binary in case of serial memories.
Using the --HWtest option and a voltmeter is possible to check that the circuit is working.
It is possible to communicate with generic I2C/SPI devices; in case of SPI it's always necessary to specify control byte and address (or addresses); the RW bit is handled automatically.
In case of problems or just for curiosity it's possible to save all data exchanged with the programmer with option -l; its optional parameter must be specified with -l=<file>; it may be a bug of getopt.
Supported languages are currently English and Italian.

Options: -BKosccal load BKosccal from file -calib load calibration from file -d, device <dev.> device -delay <ms> minimum delay [2] -ee use eeprom -err <max> max errors during writing -fuse <val> write fuse low byte (Atmel only) -fuseh <val> write fuse high byte (Atmel only) -fusex <val> write extended fuse byte (Atmel only) -h, help help -HWtest hardware test -i, info informations about programmer -i2c_r <N Ctr Addr> read N bytes from I2C bus -i2c_r2 <N Ctr Addr(2)> read N bytes from I2C bus (16b address) -i2c_r <N Ctr Addr Data> write N bytes to I2C bus -i2c_r <N Ctr Addr(2) Data> write N bytes to I2C bus (16b address) -icd <val> enable ICD (goto address) -id use ID -l, log [=file] save log -lock <val> write lock byte (Atmel only) -mode <mode> SPI mode: 00,01,10,11 -osccal loads osccal from file instead of using the value saved before erase -p, path <path> programmer path [/dev/usb/hiddev0] -pid <pid> programmer pid [0x100] -r, reserved read reserved area -rep <n> report size [64] -s, save <file> save Ihex file -se, saveEE <file> save EEPROM on Ihex file (ATxxxx only) -spi_r <N> read N bytes from SPI bus -spi_w <N Data> write N bytes to SPI bus -support supported devices -use_BKosccal copy BKosccal to osccal -v, version version -vid <vid> programmer vid [0x4D8] -w, write <file> write Ihex file -we, writeEE <file> write EEPROM on Ihex file (ATxxxx only) Examples: > op -h #help > op -d 18F2550 -ee -s read.hex #reads code and EEPROM and saves to file > op -d 16F628 -ee -w write.hex #writes > op -i2c_r 8 A0 0 #reads 8 bytes from I2C bus, control byte A0, address 0

A frequent error is to write the device name with lowercase letters instead of uppercase:
write 16F628, not 16f628

A makefile is included, so to build the application you need GCC and the maketools (MinGW/MSYS on Windows); write:
> make
Then to install it (if you wish):
> make install

Also included is Hid_test, an utility to send and receive a single 64 bit packet; it can be useful for experimenting with the hardware; in theory one could even write a complete programming script using it.


Download

Graphical interface for Windows (v0.7.9)

OpenProg is a C++ application written with Visual C++ 6 and MFC.
Using it is straightforward: just connect the programmer, start the application, select the device, load a hex file and read or write; works with XP, the V bloatware, 7.
On the "Device" tab it's possible to modify some programming options, such as ID and calibration write, use of eeprom etc.; only settings compatible with the current device will be used.
The "I2C/SPI" tab is useful for communicating with generic I2C and SPI devices; in case of I2C it's always necessary to specify the control byte (and address, if not zero); RW bit is handled automatically.
For example to write manually 3 bytes on a 24xx16 at address 64 write: A0 40 1 2 3
I didn't include an installer since there aren't any libraries and the executable is very small.
It accepts hex8 and hex32 files, and also binary in case of serial memories.
Supported languages are currently English and Italian; to add other languages it's necessary to generate the languages.rc file (from the "options" tab) and to modify it; the language name is before the respective strings enclosed in square brackets [].
Using the "Hardware Test" function and a voltmeter is possible to check that the circuit is working.
In case of problems or just for curiosity it's possible to save all data exchanged with the programmer selecting "save log file" from the "options" tab.
Command-line options are:
-d <device> , selects a target
-r <file name> , reads the target and writes to file
-w <file name> , writes a file to the target
-gui , do not exit after writing or reading (only if -w or -r are specified)

A screenshot of OpenProg:


Download application ... and sources (Visual Studio 6 workspace)

It may be of interest the fact that the DDK (driver development kit) is not required for compilation; I link explicitly to the system library hid.dll and manually load the functions needed.

GTK graphical interface for Windows and Linux (v0.7.9)

Finally, after years of waiting, a graphical interface for Linux! It's built using GTK libraries and it looks more or less like the other.
To compile it you need to install GTK libraries, GCC and the maketools (MinGW/MSYS su Windows); write:
> make
Then to install it (if you wish):
> make install

A screenshot:


Download

How to ...

Erase a device: every device is erased before being programmed; however if you still need to erase it it's sufficient to write a hex file with valid data (i.e. <0xFF) beyond the implemented memory.
For example, for PIC12-16:
:020000040000FA
:0144010000BA
:00000001FF
And for PIC18:
:020000040002F8
:020000000000FE
:00000001FF

Change Configuration Word: the easiest thing to do is to recompile sources, otherwise change the hex file; in case of PIC16 it is at address 0x2007, which is stored at 0x400E; the last byte of the line is a checksum, which can be calculated as the two's complement of the sum of all bytes in the line.
For example:
:02400E00xxxxCC , where xxxx is the new value and CC the checksum

Check that a device is blank: read it and look at displayed data; only lines with valid data are displayed, so if there are none the device is blank.

Verify that a write was successful: all write algorithms implement write verification, either during write or immediately after; if the program reports 0 errors it means that the code was succesfully verified.
If you want to check again you can read the device and compare with the original data; keep in mind that frequently not all bits are implemented; for example in PIC16 a data word is 14 bits long, so even if the source data is 0xFFFF it will be written as 0x3FFF; also the Config Words usually have some bits fixed at 0, which are not checked.

Read the reserved memory area: most PIC devices have a reserved area above the Config Words which is used for test or calibration purposes; to see it use the relative option before read; on PIC24-30-33 the executive memory is read as well.

Correctly use OscCal and other calibration words: some devices (eg. 12F5xx) store the internal oscillator calibration value in the last address of program memory and in a backup location past the ID words; after erasing all memory the control programs restore the calibration value taking it from the backup location, unless specified otherwise; other options are: use the old value (which should be the same as the backup, if none was changed before), use what is specified in the .hex file.
Similarly it's possible to overwrite the backup value with the content of the .hex file, using option "write ID and BKosccal"; in this case the ID locations will be written as well, if specified in the .hex file.
To overwrite the other calibration values use option "write Calib1 and 2".

Supported devices

I tried this programmer with a small number of devices (those I own plus some tested by other users) indicated in bold; the other devices are supported but not tested; however, considering that for every device family at least one has been verified, all of them should work without problems.
Please let me know if you verify operation with the untested devices.
Also contact me if you need other algorithms or code new ones by yourself.
Take notice:
LF series PICs are used exactly as the F ones;
Atmel AVR devices with varius suffixes are grouped together whenever the programming algorithm is exactly the same, eg. ATmega8 and ATmega8A;
EEPROM memories comprise all versions with VDDmax=5V, eg. 242LC56, 24AA256, etc.;
93x memories use 2 different algorithms, for the 93S series and for all the others, indicated with 93x (93xA have 8 bit organization).

devices supported for read and write:
10F200, 10F202, 10F204, 10F206, 10F220, 10F222,
12F508, 12F509, 12F510, 12F519, 12F609, 12F615, 12F629, 12F635, 12F675, 12F683,
12F1822, 12F1840,
16F505, 16F506, 16F526, 16F54, 16F610, 16F616, 16F627, 16F627A, 16F628, 16F628A, 16F630, 16F631, 16F636, 16F639, 16F648A, 16F676, 16F677, 16F684, 16F685, 16F687, 16F688, 16F689, 16F690, 16F716, 16F722, 16F722A, 16F723, 16F723A, 16F724, 16F726, 16F727, 16F73, 16F737, 16F74, 16F747, 16F76, 16F767, 16F77, 16F777, 16F785, 16F818, 16F819, 16F83, 16F83A, 16C83, 16C83A, 16F84, 16C84, 16F84A, 16C84A, 16F87, 16F870, 16F871, 16F872, 16F873, 16F873A, 16F874, 16F874A, 16F876, 16F876A, 16F877, 16F877A, 16F88, 16F882, 16F883, 16F884, 16F886, 16F887, 16F913, 16F914, 16F916, 16F917, 16F946,
16F1516, 16F1517, 16F1518, 16F1519, 16F1526, 16F1527, 16F1823, 16F1824, 16F1825, 16F1826, 16F1827, 16F1828, 16F1829, 16F1847, 16LF1902, 16LF1903, 16LF1904, 16LF1906, 16LF1907, 16F1933, 16F1934, 16F1936, 16F1937, 16F1938, 16F1939, 16F1946, 16F1947,
18F242, 18F248, 18F252, 18F258, 18F442, 18F448, 18F452, 18F458, 18F1220, 18F1230, 18F1320, 18F1330, 18F13K50, 18F14K50, 18F2220, 18F2221, 18F2320, 18F23K20, 18F2321, 18F2331, 18F2410, 18F24J10, 18F24J11, 18F2420, 18F24K20, 18F2423, 18F2431, 18F2439, 18F2450, 18F24J50, 18F2455, 18F2458, 18F2480, 18F2510, 18F25J10, 18F25J11, 18F2515, 18F25K20, 18F2520, 18F2523, 18F2525, 18F2539, 18F2550, 18F25J50, 18F2553, 18F2580, 18F2585, 18F2610, 18F26J11, 18F26J13, 18F2620, 18F26K20, 18F26J50, 18F26J53, 18F2680, 18F2682, 18F2685, 18F27J13, 18F27J53, 18F4220, 18F4221, 18F4320, 18F43K20, 18F4321, 18F4331, 18F4410, 18F44J10, 18F44J11, 18F4420, 18F44K20, 18F4423, 18F4431, 18F4439, 18F4450, 18F44J50, 18F4455, 18F4458, 18F4480, 18F4510, 18F45J10, 18F45J11, 18F4515, 18F4520, 18F45K20, 18F4523, 18F4525, 18F4539, 18F4550, 18F45J50, 18F4553, 18F4580,18F4585, 18F4610, 18F46J11, 18F46J13, 18F4620, 18F46K20, 18F46J50, 18F46J53, 18F4680, 18F4682, 18F4685, 18F47J13, 18F47J53, 18F8722,
24F04KA200, 24F04KA201, 24F08KA101, 24F08KA102, 24F16KA101, 24F16KA102, 24FJ16GA002, 24FJ16GA004, 24FJ32GA002, 24FJ32GA004, 24FJ48GA002, 24FJ48GA004, 24FJ64GA002, 24FJ64GA004, 24FJ64GA006, 24FJ64GA008, 24FJ64GA010, 24FJ96GA006, 24FJ96GA008, 24FJ96GA010, 24FJ128GA006, 24FJ128GA008, 24FJ128GA010, 24FJ32GA102, 24FJ32GA104, 24FJ32GB002, 24FJ32GB004, 24FJ64GA102, 24FJ64GA104, 24FJ64GB002, 24FJ64GB004, 24FJ64GB106, 24FJ64GB108, 24FJ64GB110, 24FJ128GA106, 24FJ128GB106, 24FJ128GA108, 24FJ128GB108, 24FJ128GA110, 24FJ128GB110, 24FJ192GA106, 24FJ192GB106, 24FJ192GA108, 24FJ192GB108, 24FJ192GA110, 24FJ192GB110, 24FJ256GA106, 24FJ256GB106, 24FJ256GA108, 24FJ256GB108, 24FJ256GA110, 24FJ256GB110, 24HJ12GP201, 24HJ12GP202, 24HJ16GP304, 24HJ32GP202, 24HJ32GP204, 24HJ32GP302, 24HJ32GP304, 24HJ64GP202, 24HJ64GP204, 24HJ64GP206, 24HJ64GP210, 24HJ64GP502, 24HJ64GP504, 24HJ64GP506, 24HJ64GP510, 24HJ128GP202, 24HJ128GP204, 24HJ128GP206, 24HJ128GP210, 24HJ128GP306, 24HJ128GP310, 24HJ128GP502, 24HJ128GP504, 24HJ128GP506, 24HJ128GP510, 24HJ256GP206, 24HJ256GP210, 24HJ256GP610,
30F1010, 30F2010, 30F2011, 30F2020, 30F2023, 30F2012, 30F3010, 30F3011, 30F3012, 30F3013, 30F3014, 30F4011, 30F4012, 30F4013, 30F5011, 30F5013, 30F5015, 30F5016, 30F6010, 30F6011, 30F6012, 30F6013, 30F6014, 30F6015,
33FJ06GS101, 33FJ06GS102, 33FJ06GS202, 33FJ12GP201, 33FJ12GP202, 33FJ12MC201, 33FJ12MC202, 33FJ16GP304, 33FJ16GS402, 33FJ16GS404, 33FJ16GS502, 33FJ16GS504, 33FJ16MC304, 33FJ32GP202, 33FJ32GP204, 33FJ32GP302, 33FJ32GP304, 33FJ32GS406, 33FJ32GS606, 33FJ32GS608, 33FJ32GS610, 33FJ32MC202, 33FJ32MC204, 33FJ32MC302, 33FJ32MC304, 33FJ64GP202, 33FJ64GP204, 33FJ64GP206, 33FJ64GP306, 33FJ64GP310, 33FJ64GP706, 33FJ64GP708, 33FJ64GP710, 33FJ64GP802, 33FJ64GP804, 33FJ64GS406, 33FJ64GS606, 33FJ64GS608, 33FJ64GS610, 33FJ64MC202, 33FJ64MC204, 33FJ64MC506, 33FJ64MC508, 33FJ64MC510, 33FJ64MC706, 33FJ64MC710, 33FJ64MC802, 33FJ64MC804, 33FJ128GP202, 33FJ128GP204, 33FJ128GP206, 33FJ128GP306, 33FJ128GP310, 33FJ128GP706, 33FJ128GP708, 33FJ128GP710, 33FJ128GP802, 33FJ128GP804, 33FJ128MC202, 33FJ128MC204, 33FJ128MC506, 33FJ128MC510, 33FJ128MC706, 33FJ128MC708, 33FJ128MC710, 33FJ128MC802, 33FJ128MC804, 33FJ256GP506, 33FJ256GP510, 33FJ256GP710, 33FJ256MC510, 33FJ256MC710,
2400, 2401, 2402, 2404, 2408, 2416, 2432, 2464, 24128, 24256, 24512, 241024, 241025,
25010, 25020, 25040, 25080, 25160, 25320, 25640, 25128, 25256, 25512, 251024,
93S46, 93x46, 93x46A, 93S56, 93x56, 93x56A, 93S66, 93x66, 93x66A, 93x76, 93x76A, 93x86, 93x86A,
AT90S1200, AT90S2313, AT90S8515, AT90S8535, ATmega48, ATmega8, ATmega88, ATmega8515, ATmega8535, ATmega16, ATmega164A, ATmega168, ATmega32, ATmega324A, ATmega328, ATmega64, ATmega644A, ATmega1284,
ATtiny12, ATtiny13, ATtiny24, ATtiny26, ATtiny261, ATtiny2313, ATtiny44, ATtiny48, ATtiny461, ATtiny4313, ATtiny84, ATtiny88, ATtiny861

devices supported for read only:
12C508, 12C508A, 12C509, 12C509A, 12C671, 12C672, 12CE673, 12CE674

Important!! Don't use 3.3V devices without the 3.3V expansion boards, otherwise permanent damage may occur; the software verifies that such adapters are present before starting to program, but obviously you need to select the proper device; the 3.3V devices are:
16F1xxx,18FxxJxx,18FxxKxx,24Fxxx,24Hxxx,33Fxxx.
Also don't put any 24F-33F on the 30F socket, which works at 5V.

Communication protocol

To design a communication protocol we must take into account some often contrasting requirements:
transfer speed and efficiency, code size, adaptability and expandability.
Differently from serial links, USB is packet based; a packet is received altogether, but HID devices can only exchange them every ms, so it is out of question to manage timings directly as with serial ports.
It's necessary to introduce synthetic commands that the microcontroller can use to recreate the proper waveforms.
Furthermore, one objective of a reliable programmer is to be independent from the host speed and CPU occupation, so the task of generating waveforms would anyways be given to the microcontroller.
In general we can find two types of programmers: simple serial programmers only take commands to change voltage levels; host software manages both timings and programming algorithms but needs all the CPU time and is dramatically affected by other processes running on the system.
At the other extreme are "smart" programmers, which autonomously manage timings and algorithms, but must be updated to support new devices and tend to require much memory to store code.
I chose a combination of both: ICSP (In Circuit Serial Programming) commands are implemented in firmware, but the host software manages the algorithms.
In order to increase speed and efficiency some instructions correspond to sequences of frequently repeated commands, such as sequential reads.
The advantage of this approach is that timings are very precise, while the extreme variety of algorithms does not increase the firmware code size.
Another advantage is that once the basic commands are verified the development of algorithms is done on the PC, without the need for frequent firmware updates.
For example, this is the sequence used to enter program mode for 16F628 and read DevID:

SET_PARAMETER //set delays to be used by other instructions SET_T1T2 //T1 & T2 1 //T1=1us 100 //T2=100us EN_VPP_VCC //Vpp & Vcc = 0 0x0 SET_CK_D //Clock and Data as output and 0 0x0 EN_VPP_VCC //Vpp enabled 0x4 NOP //small delay EN_VPP_VCC //Vdd+Vpp enabled 0x5 NOP //small delay LOAD_CONF //program counter to 0x2000 0xFF //fake config 0xFF //fake config INC_ADDR_N //increment address by 6 0x06 READ_DATA_PROG //read DevID ...

In addition to ICSP commands other instructions manage the programmer, control programming voltages, execute precise delays, communicate via I2C or SPI bus.
Every instruction is executed in at least 40 us, due to the interpreter loop execution time.
ICSP commands use T1 or T2 as values for delays; all instructions return an echo, with the exception of FLUSH, which immediately sends the output buffer and stops the execution of current packet.
In case an instruction doesn't have enough parameters it returns an error (0xFE) and the execution of current packet is halted.
The state of USB connection is signaled by LED2: it blinks at 4 Hz during enumeration, at 1 Hz in normal operation.
LED1 shows when there are instructions being executed.
Following is the list of all instructions:

Instruction	Value	Parameters	Answer	Notes
NOP	0x00	none	echo	no operation
PROG_RST	0x01	none	echo + 10B	programmer reset; sends fw version (3B), ID (3B), " RST" string
PROG_ID	0x02	none	echo+ 6B	sends fw version (3B), ID (3B)
CHECK_INS	0x03	1B	echo + 1B	if specified instruction exists returns its code, otherwise returns error (0xFE)
FLUSH	0x04	none	none	flushes output buffer (sends 64B) and stops command interpreter for the current packet.
VREG_EN	0x05	none	echo	turns on the voltage regulator
VREG_DIS	0x06	none	echo	turns off the voltage regulator
SET_PARAMETER	0x07	1B	echo	sets internal parameters; byte1: parameter to change, byte 2-3 value: SET_T1T2 (=0):      T1 &  T2 SET_T3 (=1):           T3(H,L) SET_timeout (=2):    timeout(H,L) SET_MN (=3):         M, N
WAIT_T1	0x08	none	echo	waits T1 us (1us default)
WAIT_T2	0x09	none	echo	waits T2 us (100us default)
WAIT_T3	0x0A	none	echo	waits T3 us (2ms default)
WAIT_US	0x0B	1B	echo	waits N us
READ_ADC	0x0C	none	echo +2B	reads regulator voltage (effective 10bits, MSB-LSB); considering the input divider, voltage in V is <value>/1024*5*34/12
SET_VPP	0x0D	1B	echo +1B	sets regulator voltage to <parameter>/10; if error is < 200 mV within 15ms returns <parameter>, otherwise error (0xFE)
EN_VPP_VCC	0x0E	1B	echo	controls Vpp and Vcc on the programmed device; 1 bit for level (0-1), 1bit for impedance (keep at 0); bit 0-1: Vcc, bit 2-3: Vpp
SET_CK_D	0x0F	1B	echo	controls CK, D, PGM on the programmed device; 1 bit for level (0-1), 1bit for impedance (low-high); bit 0-1: D, bit 2-3: CK, bit 4-5: PGM
READ_PINS	0x10	none	echo +1B	reads the state of control lines, 1 bit level (0-1), 1bit impedence (low-high); bit 0-1: D, bit 2-3: CK, bit 4-5: PGM
LOAD_CONF	0x11	2B	echo	ICSP command: Load configuration (000000), T1 us between command and data; 14 bit data (right aligned, MSB-LSB)
LOAD_DATA_PROG	0x12	2B	echo	ICSP command: Load Data in Program Memory (000010), T1 us between command and data; 14 bit data (right aligned, MSB-LSB)
LOAD_DATA_DATA	0x13	1B	echo	ICSP command: Load Data in Data Memory (000011), T1 us between command and data; 8 bit data
READ_DATA_PROG	0x14	none	echo +2B	ICSP command: Read Data from Program Memory (000100), T1 us between command and data; 14 bit data (right aligned, MSB-LSB)
READ_DATA_DATA	0x15	none	echo +1B	ICSP command: Read Data from Data Memory (000101), T1 us between command and data; 8 bit data
INC_ADDR	0x16	none	echo	ICSP command: Increment Address (000110), T1 us delay at the end
INC_ADDR_N	0x17	1B	echo	ICSP command: Increment Address (000110), T1 us delay at the end; repeated N times
BEGIN_PROG	0x18	none	echo	ICSP command: Begin Programming (001000)
BULK_ERASE_PROG	0x19	none	echo	ICSP command: Bulk Erase Program Memory (001001)
END_PROG	0x1A	none	echo	ICSP command: End Programming (001010)
BULK_ERASE_DATA	0x1B	none	echo	ICSP command: Bulk Erase Data Memory (001011)
END_PROG2	0x1C	none	echo	ICSP command: End Programming (001110)
ROW_ERASE_PROG	0x1D	none	echo	ICSP command: Row Erase Program Memory (010001)
BEGIN_PROG2	0x1E	none	echo	ICSP command: Begin Programming (0011000)
CUST_CMD	0x1F	1B	echo	ICSP command specified in the parameter
PROG_C	0x20	2B	echo +1B	Programs a word following 12Cxxx algorithm: 000010, 001000, 001110, M pulses & N overpulses
CORE_INS	0x21	2B	echo	PIC18 ICSP command: Core instruction (0000); 16 bit data (MSB-LSB)
SHIFT_TABLAT	0x22	none	echo +1B	PIC18 ICSP command: Shift TABLAT (0010); 8 bit data
TABLE_READ	0x23	none	echo +1B	PIC18 ICSP command: Table read (1000); 8 bit data
TBLR_INC_N	0x24	1B	echo+N+NB	PIC18 ICSP command: Table read post-inc (1001); 8 bit data; repeats N times; returns N and NB data
TABLE_WRITE	0x25	2B	echo	PIC18 ICSP command: Table write (1100); 16 bit data (MSB-LSB)
TBLW_INC_N	0x26	(2N+1)B	echo	PIC18 ICSP command:  Table write post-inc (1101); 16 bit data (MSB-LSB); repeats N times (N is the first parameter)
TBLW_PROG	0x27	4B	echo	PIC18 ICSP command: Table write and program (1111); 16 bit data (MSB-LSB); also executes a NOP with a delay specified in  parameters 3-4 (in us)
TBLW_PROG_INC	0x28	4B	echo	PIC18 ICSP command: Table write and program post-inc (1110); 16 bit data (MSB-LSB); also executes a NOP with a delay specified in  parameters 3-4 (in us)
SEND_DATA	0x29	3B	echo	PIC18 ICSP command specified in byte 1; sends 16 bit data (MSB-LSB)
READ_DATA	0x2A	1B	echo+1B	PIC18 ICSP command specified in byte 1; reads 8 bit data
I2C_INIT	0x2B	1B	echo	Initializes I2C communication: 0xFF disables I2C; bit 6: slew rate control for speed > 100kbps; bit 5:3 speed: 0=100k, 1=200k, 2=400k, 3=800k, 4=1M; attention: use pull-up resistors according to selected speed; bit 2:0: logic level of A2-A1-A0 (on device)
I2C_READ	0x2C	3B	echo+1+NB	Reads <parameter 1> bytes from I2C bus using <parameter 2> as control byte and <parameter 3> as address; forces automatically the RW bit in the control byte. Responds with <parameter 1> + Data or with ACK_ERR (0xFD) in case of acknowledge error (eg. if there are no devices on the bus)
I2C_WRITE	0x2D	3B+NB	echo	Writes <parameter 1> bytes to I2C bus using <parameter 2> as control byte and <parameter 3> as address; forces automatically the RW bit in the control byte. Responds with ACK_ERR (0xFD) in case of acknowledge error (eg. if there are no devices on the bus). For 2 byte addresses just use the 2nd byte of address as the first byte of data.
I2C_READ2	0x2E	4B	echo+1+NB	Reads from I2C bus; identical to I2C_READ, but uses 2 bytes for addressing
SPI_INIT	0x2F	1B	echo	Initializes SPI communication: 0xFF disables SPI bit 1:0 speed: 0=100kbps, 1=200kbps, 2=300kbps, 3=500kbps bit 2:3 SPI mode
SPI_READ	0x30	1B	echo+1+NB	Reads <parameter 1> bytes from SPI bus. Returns <parameter 1> + Data; If <parameter 1>=0 returns the byte that was last received (during either read or write)
SPI_WRITE	0x31	1B+NB	echo+1B	Writes <parameter 1> bytes to SPI bus.
EXT_PORT	0x32	2B	echo	Forces levels of communication ports: <parameter 1> = <RB7:RB0> <parameter 2> = <RC7,RC6,RA5:RA3> Does not change signal direction.
AT_READ_DATA	0x33	3B	echo+1+2NB	ATMEL command: read program memory (0010H000); reads <parameter 1> words at address <parameter 2> : <parameter 3> via SPI. Returns <parametro 1> + Data
AT_LOAD_DATA	0x34	3B+2N	echo+1B	ATMEL command: load program memory page (0100H000); loads <parameter 1> words at address <parameter 2> : <parameter 3> via SPI. Returns <parameter 1>
CLOCK_GEN	0x35	1B	echo	Generates a clock signal on RB3 (using CCP1-2 and TIMER1) bit 2:0 frequency: 0=100kHz, 1=200kHz, 2=500kHz, 3=1MHz, 4=2MHz, 5=3MHz, 6=6MHz; Disables PWM1 output (DCDC regulator)
SIX	0x36	3B	echo	PIC24 ICSP command: Core instruction (0000); 24 bit data (MSB first)
SIX_LONG	0x3E	3B	echo	PIC24 ICSP command: Core instruction (0000); 24 bit data (MSB first); appends 2 ICSP_NOP at the end
SIX_N	0x3F	1+3NB	echo	PIC24 ICSP command: Core instruction (0000); N * 24 bit data (MSB first); sends N instructions and adds M ICSP_NOP after each one; N=<parameter1>&0x3F, M=<parameter1> >>6
REGOUT	0x37	none	echo +2B	PIC24 ICSP command: Shift out VISI register (0001); 16 bit data
ICSP_NOP	0x38	none	echo	PIC24 ICSP command: Execute NOP (0000)
TX16	0x39	1+2NB	echo+1B	Transmits 16 bit words over ICSP (MSB first); clock period: 2*(T1-1)+1 us
RX16	0x3A	1B	echo+1+2NB	Receives 16 bit words over ICSP (MSB first); clock period: 2*(T1-1)+1 us
uW_INIT	0x3B	none	echo	Initializes MicroWire communication
uW_TX	0x3C	1B+NB	echo+1B	Writes <parameter 1> bits to MicroWire bus. Data is specified MSB first
uW_RX	0x3D	1B	echo+1+NB	Reads <parameter 1> bits from MicroWire bus. Returns <parameter 1> + Data, MSB first
READ_RAM	0xF0	2B	echo+3B	Reads from host memory; 16 bit address, 8 bit data; echoes address
WRITE_RAM	0xF1	3B	echo+3B	Writes to host memory; 16 bit address, 8 bit data; echoes address and data
LOOP	0xF2	none	none	Resets instruction pointer and executes all instructions again. For test purposes only.

The circuit (v1.7)

The project is based on a 28 pin 18F2550, but only about a third of the memory and 0% of eeprom was used, so it will fit confortably into the smaller 2455.
The 2458 and 2553 have a 12 bit ADC, so only recompilation is required.
With some modifications I adapted the code to the 2450; since this model lacks the MSSP module I used a software implementation of I2C and SPI; it also lacks the second PWM channel, therefore it can't generate the clock for Atmel chips (for those that are configured with external clock); in this case RB3 can be used to turn on an external oscillator (which would be inserted in a modified Atmel expansion board).
The use of the corresponding 40 pin devices (4450, 4455, 4458, 4550, 4553) requires modification of the PCB.
In order to implement an USB pheripheral with a PIC micro we need very few components: the main microcontroller, a quartz, some capacitors, and a USB type B receptacle, exactly as written in application notes from Microchip.
To be able to program PIC devices we need two digital lines for clock and data and two supply voltages, VCC and VPP, which are controlled using three transistors; VPP comes from a switching voltage regulator formed by Q4, L1, D3 which is described later.

Schematic diagram of base module:


PCB of base module:

Many components are optional, are only needed to program some types of devices or for future applications: expansion connectors CONN2-3, protection resistors R11:23 (considering their cost why not use them?), I2C pull-up resistors R26-27, S1 switch, ICSP-IN CONN4 (right now it's used to program the main microcontroller without extracting it).
The pcb was optimized to fit the solder side, however a few jumpers are needed on the component side; if you want you can avoid that by using a double side pcb.
Pay attention to the orientation of transistors: Q1's emitter to the left, Q2 up, Q3 and Q4 right.
Once connected to the PC, a working circuit blinks D2 at 4Hz until the enumeration is completed, then at 1Hz.
To verify that everything is working correctly use the "Hardware Test" function in the control program: in this mode, to be executed without target devices, all the outputs (CK, D, PGM, VDDU, VPPU, which can be reached on pins 14-15-12-1-4 of U3, with respect to GND, pin 5) are activated in various combinations; if the measured voltages correspond to what is presented on screen then the hardware is correctly assembled.
VPP voltage could be different from the set value by up to 1V; this is due to the fact that the DCDC converter takes VCC as reference voltage; the latter comes from the USB cable and can vary from 4.75V to 5.25V; in addition the feedback voltage divider (R1-R2) can introduce another 5% of error.
The most common causes of malfunction are:
incorrect orientation of transistors,
incorrect inductor value,
pcb defects like shorts or opens,
unsoldered capacitors,
main microcontrollorer not programmed.


Component list:
U1 12Mhz quartz (also 4, 8, 16, 20; reconfiguration of input divider options is required)
U2 18F2550 (also 2450,2455,2458,2553,4450,4455,4458,4550,4553)
U3 20p socket.
U4 8p socket.
Q1-2 BC557 (or any PNP, pay attention to polarity)
Q3-4 BC547 (or any NPN, pay attention to polarity)
D1-2 LED
D3 1N4148 (or any diode, better if Shottky)
L1 100uH resistor type or other
R1 22K
R2 12K
R3 100K
R4:6 10K
R7 1M
R8-9 2.2K
R10 10K
R11:23 100
R24-25 300K
R26-27 10K
C1 22-100uF 25V
C2-3 22pF
C4 >= 220nF
C5 100nF
C6 10uF
C7-8 100nF
CONN1 USB type B
CONN2-3 10 pin female stripline
CONN4 5 pin stripline

How to use

The basic circuit can host PIC devices with 8, 14, 18, and 20 pins (except 10Fxxx); they should be inserted in U3 with alignment to pin1:





I2C memories go in U4.
I plan to make an adapter for 10Fxxx with 6 or 8 pins; in the meantime it's possible to get by using wires.
Other devices can be programmed using expansion boards plugged to connectors CONN2-3:

• 28-40 pin PICs + ICSP connector
• 8-20 pin PICs (same as main board, but there's more space for a ZIF socket) + ICSP conn.
• 3.3V PIC16-18 + ICSP conn. (this board has also a 3.3V regulator)
• PIC24-30-33 + ICSP conn. (this board has also a 3.3V regulator)
• I2C, SPI, uW memories and I2C-SPI conn.
• 8-14-20-28-40 pin ATMEL micros and I2C-SPI conn
• ST72 (future)

Of course they're not required for basic operation.
In assembling the adapters I suggest to insert the expansion connectors from the component side, and keep their plastic spacer on that side; this improves the solder strength, especially during extraction.
In case of 3.3V devices the presence of a 3.3V adapter (which has RB1 and RB0 shorted) is checked by the software in order to avoid irreversible damage.
The 16F1xxx could be programmed also without such adapter (only the 16LF1xxx need it), but the check is present for both so there is no chance of confusion.
Due to the impossibility of erasing dsPIC30 protection registers at 3.3V, these devices are now powered at 5V; who already made the adapter (prior to v. 1.7) can easilymodify it.
The following images show how to insert various target devices in the expansion boards:
     
Smaller devices have to be aligned to pin 1 of the respective socket.
Target chips can also be programmed in circuit by routing ICSP signals (VPPU,VDDU,ICD,ICK,GND) to the application board; these signals are present in the base module expansion connectors or in some expansion boards as a discrete connector; note that low voltage devices require ICSP signals from a low voltage expansion board.
ICSP-IN connector is used to program the main micro without extracting it, by means of another programmer.

Pin mapping of various connectors in the base and expansion boards:


Map of resources used:

Pin	Various functions	ICSP	I2C-EEPROM	SPI-EEPROM	SPI-ATMEL	uW-EEPROM
RB7		PGM
RB6		ICSP clock
RB5		ICSP data	A2			W (6)
RB4			A1	HLD		S (1)
RB3			A0	CS	Device clock	PRE (7)
RB2	expansion
RB1			Clock	Clock	SPI Clock	Clock
RB0			Data	Data out (MOSI)	Data out (MOSI)	Data out
RC7				Data in (MISO)	Data in (MISO)	Data in
RC6			WP	WP	RESET
RC5	USB D+
RC4	USB D-
RC2	DCDC PWM
RC1	controls VDD
RC0	controls VPP
RA5	expansion
RA4	expansion
RA3	expansion
RA2	LED 2
RA1	LED 1
RA0	ADC for regulator
RE3	S1 switch

The schematic diagram was drawn with Gschem, an open source program that comes with GEDA suite.
In their website it's not evident (as they all use linux), but it's also possible to run the program in windows under cygwin; I suggest to use the latest version (you need to compile sources).
PCBs were drawn with PCB; in this case there is also a (somewhat limited) windows version.
With a little effort the circuit can also be mounted on experimental boards, without pcb.

Schematic diagram of base module: .pdf, .png; expansion boards: .pdf; everything in gschem format: Oprog.sch
Pcb of base module: .pdf, .png; base module + expansion boards: .pdf, .png; everything in PCB format: Oprog.pcb
Complete archive, includes sources, gerber, pdf, png

How to program the main micro the first time?

This is an interesting problem: a new device can't work as programmer, so it must be programmed in some way.
Apart from asking someone else to do it for you, my advice is to build one of  those serial programmers, like JDM, to do the job the first time.
It may seem strange to use a programmer to build another one, but there is no way to interface USB without firmware; I think the effort is worth it because serial programmers are not very reliable, are slow, and of course not portable to new computers that lack serial ports.
It would also be a good idea to buy a backup micro, in order to program it with updated firmware versions.


The main circuit and some expansion boards (28-40p PIC, 8-20p PIC with ZIF, 3.3V PIC16-18, 3.3V PIC24-30-33, EEPROM, ATMEL):

Switching voltage regulator

In order to generate a voltage higher than 5V we need a boost switching converter.
On the market there are thousands of single chip solutions, but I used instead the microcontroller itself and a few external components.
The width of output pulses will vary to keep the output voltage stable over all operating conditions.
In practice this is a digitally controlled regulator, as shown in the following diagram:



The ADC converter presently uses the 5V supply as a reference, so the output voltage will follow it; it is possible to connect an external reference to RA3 to improve the overall precision.
Switching frequency is 90 kHz, which is well over the cutoff frequency of the output LC filter (~2,3 kHz).
The performance is limited by losses due to the transistor, diode, inductor, but since the load is very low we can use low-cost (even recycled) components; to improve load capability switch to a better transistor, a Shottky diode, a higher rated inductor.
Anyways, in order to design a suitable regulator (block C above) it's necessary to work in s domain and model the converter itself; this has fortunately been done already, some info is available for example here.
With present component values the boost converter operates in dicontinuous mode; critical current is:
Icrit=Vu*T/(16*L)=86 mA
well over expected load, supposed to be 1 mA.
Other parameters:
Vi=5
Vu=12.5
D=(Vu-Vi)/Vo
L=100e-6
C=47e-6
I=1e-3
R=12/I
Rl=1.6 (inductor series resistance)
T=1/90e3

vu 1 vu M-1 Vu 2M -1
--- = Gdo ---------- where Gdo = 2 --- ------- , M = ---, wp = ----------
D 1 + s/wp D 2M -1 Vi (M-1) RC

Transfer function results to be:

vu 127.58
-- = -------------
D 0.22031 s + 1

Which has the following Bode diagram:


It seems that the system, in closed loop, would be stable even by itself; however it would have a steady state error of 1/DCgain.
It's better to use a controller with a pole on the origin and a zero to stabilize everything, for example the following controller:

D 0.25 (s + 50)
C = --- = -------------
err s

Overall transfer function would be:

vu 144.77 s + 7238.4
-- = -----------------
vi s² + 4.539 s

The system is stable, with a phase margin of ~75º.
Since we operate in the digital domain we must choose the sampling frequency.
It can't be too high because of execution speed; if too low it limits the regulator bandwidth; a period of 250 us was a good compromise.
The various transfer functions are converted to z domain using bilinear transformation:

vu 0.018199 z²+ 0.00022607 z - 0.017973
-- = ------------------------------------
vi z² - 1.9989 z + 0.99887

The controller is:

D 0.25156*z - 0.24844 C1 - C2 z^-1
C = --- = ------------------- = -----------
err z - 1 1 -z^-1

Remember that z^-1 represents a delay of one clock cycle.
Next we must deal with quantization and calculation errors.
A/D converter is 10 bits wide, and is triggered by timer2; at the end of conversion an interrupt calls the regulation routine, which calculates the new duty cycle for the PWM peripheral, also 10 bits wide.
On the feedback path it's necessary to include a voltage divider in order to limit ADC input voltage to [0,5V]; R1 and R2 do this.
So the block diagram is modified as follows:


a=12/34

Vu=C'H(Vi-aVu)

Vu C'H
-- = ------
Vi 1+aC'H

To compare with the previous model we can multiply both terms by a; simply remembering to change the set point we can decide that the new input is Vi/a, and equate with the previous expression:

Vu aC'H CH
---- = ------ = ----
Vi/a 1+aC'H 1+CH

aC'=C

aC1' - aC2' z^-1 C1 - C2 z^-1
aC'= --------------- = C = ------------
1 - z^-1 1 - z^-1

aC1'=C1
aC2'=C2

Since the hardware works with 10 bit digital data we can go from D/err to pwm/[err]:

[err]=err*1024/5
pwm=D*1024

D pwm/1024 pwm C1' - C2'z^-1
C'= --- = ------------ = ------- = ------------
err [err]/1024*5 [err]*5 1 -z^-1

pwm(1 - z^-1)=[err](5*C1/a - 5*C2/a z^-1)=[err](3.564 - 3.52 z^-1)

It's clear that integer multiplications can't be used with these coefficients; the easiest solution is to work with fractional values (i.e. divide output by 2^N and multiply coefficients accordingly); considering that pwm output is 10 bits wide and left-aligned, we can easily work with values divided by 64.

pwm(1 - z^-1)=[err](k1 - k2 z^-1)/64

k1=5C1/a*64=228.12 ~ 228
k2=5C2/a*64=225.25 ~ 225

Following are step responses of continuous-time system (blue), discrete-time system (red), discrete-time system with approximate cefficients (green); As you can see they're almost coincident.

For all calculations I used Octave, an open source mathematical modeling tool; version 3 has just been released, and it can be used also under the famous windows (almost)operating system.
If someone is interested these are the modeling scripts I used.
The real code for the control function was written in assembly; this is necessary for performance reasons.
In fact our C compiler calls a library function to perform multiplications, so it has to save many variables on the stack causing a delay; in this case the resulting execution time had reached 50 us, which is a significant fraction of the sampling period.
Instead, by avoiding function calls and manually coding the 16x8 bit multiplication (see k1 & k2), the execution time is down to 12 us.

Some real waveforms:

 
Power-up transient, 50 ms/div


Step response to load change (load on top trace, AC coupled output on bottom tr.), 50ms/div


Step response to set point change (from 11,5 to 12,5 V), 50 ms/div

How to contribute

The best way to contribute to this project is to build it, use it, and report bugs or suggestions.
Also there are still many devices to test; check the list in supported devices.
Whoever has the know-how and patience can also expand support to other devices.
Or if you have a device that is not supported you can send it to me so that I can work on it.
If you find this project useful write me a couple of lines: , and if you modified it show me your work.
On SourceForge you can find some forums where you can discuss about this project.

Downloads

Schematic diagram and pcb: complete archive.
Firmware: complete MPLAB project or compiled firmware (.hex) or a version for 18F2450 (with reduced functionality, see the circuit).
OP (command-line for Linux & Windows)
OpenProg (windows): application only;   sources (Visual Studio 6 workspace)
opgui (GTK GUI for Linux & Windows)
Octave scripts

History

a long time ago	need for a reliable and free USB programmer
2007	experiments with USB PIC and varionus firmwares; voltage regulator
2008	first prototypes and software
july 2008	documentation and website, released version 0.3
august 2008	version 0.4: added support for I2C EEPROMs
november 2008	version 0.5: I2C & SPI bus, added some ATMEL devices
january 2009	control progs v0.5.1: added some PIC devices, removed some bugs
march 2009	control progs v0.5.2 and v0.5.3: added some PIC an Atmel devices, removed some bugs
april 2009	schematic diagram and pcb v1.4: changed Atmel adapter
june 2009	version 0.6: fully GPL2 USB firmware, added 93Sx6 MicroWire EEPROMs
september 2009	version 0.6.1: solved some SPI bugs, added some Atmel devices and 93Cx6C
october 2009	control progs v0.6.2: bugfix
january 2010	version 0.7.0: added PIC24 and SPI EEPROMs; circuits v1.5: expansion board for 3.3V PIC24-30-33
february 2010	control progs v0.7.1: added some PIC18 and Atmel devices; bugfix
march 2010	control progs v0.7.2: added some MicroWire eeproms; bugfix
april 2010	control progs v0.7.3: added PIC16F1xxx; OP works in windows; code rework and bugfix; circuits v1.6: expansion board for 3.3V PIC16-18
may 2010	control progs v0.7.4: added PIC18FxJx, PIC18FxKx, PIC24H, dsPIC30-33, ATtiny2313, 241024; bugfix; circuits v1.7: modified PIC24-30-33 expansion board, PIC30 now at 5V
june 2010	control progs v0.7.5: corrected write bug for 18Fx config, added "goto ICD" write for 16Fx, various minor fixes
july 2010	firmware v0.7.6: modified TX16 and RX16 with variable period for communication with ICD; control progs v0.7.6: byte by byte read with 93xx6 for better compatibility; first release of pdb debugger (v0.1);
august 2010	control progs v0.7.7: added 16F72x, some minor fixes; circuits v1.7.1: reduced value of R173 on the PIC16/18 LV expansion board
april 2011	control progs v0.7.8: updated algorithms for 16F87xA and 16F62xA, 16F1822 becomes 12F1822, added 30F1010, 30F2020, 30F2023, 16F1847, 12F1840 16F1516, 16F1517, 16F1518, 16F1519, 16F1526, 16F1527; greater modularity for source code, corrcted some bugs
june 2011	control progs v0.7.9: added ATtiny11-12-24-26-261-44-48-461-4313-84-88-861, ATmega48-88-164A-168-324A-328-644A-1284, 16LF1902-3-4-6-7, various minor modifications and fixes; new graphical interface based on GTK for Linux and Windows
the future	increase support for PIC and ATMEL micros (as soon as I can get free samples); add ST72, JTAG, parallel memories; expand ICD support; compile firmware also with SDCC