Apple iPod on Linux - Archive, 2002
Activities
- 7-3
-
tex9 is developing a plug-in to xtunes called xpod to support drag-and-drop of songs and playlists onto an iPod. I have no idea how they're going to access the iPod filesystem, but it sounds interesting. They need beta-testers, so I'm going to sign up and learn more.
- 5-22
- 3-16
- I'm trying to get VMWare 3.0 (running WinNT4.0) to pick up iPod as a generic SCSI device. Then I can use MacDrive to read/write it.
Sounds good, but it doesn't work. I get this error from the VMWare log:
Mar 16 11:12:18: scsi0:1|SCSI PASS-THRU: OPEN /dev/sg0
Mar 16 11:12:18: scsi0:1|SCSI PASS-THRU: Version 3.1.20 Mem Rsvd 32 KB
Mar 16 11:12:18: scsi0:1|scsi0:1: ioctl(/dev/sg0, SG_SET_TRANSFORM)
unexpectedly failed. "Invalid argument"
Mar 16 11:12:18: VMX|Msg_Post Error: msg.device.badconnect
Failed to connect device scsi0:1
Argh, foiled at every turn. Does anyone have any ideas? My kernel is now 2.4.9-31.
- 1-22
- Mattes wrote in to advise that I may be using an unsupported Firewire card.
He pointed to this list of supported PCMCIA cards.
He's right of course - mine doesn't appear on the list. But it *does* appear on the list of compatible cards for the
Firewire project, and even more, it Works Great.
But there were some interesting notes about configuration of the card in /etc/pcmcia/config that I'd never noticed.
So I'm wondering if some of my problems are due to a misconfigured PCMCIA card. I'm hoping it's that and not incompatibility,
because it's too late to return it. It does raise an interesting question: what IEEE1394 cards are people using
successfully with iPod/Linux?
- Paul told me that he's had the same exact problems getting hfsplusutils to read iPod,
and suggested I tweak Mark's SBP2 hack even further to force every device to become a disk. Works for him, but not for me.
- FonkiE enhanced his Perl script to parse playlists as well as songs,
and wrote a new script to parse the EQ settings file.
He's interested in learning the min/max of each band (>=655).
He's also still in search of sample iTunesDB to help him with his parsing work.
- I didn't have much luck with 2.4.17, and the newer version of the compiler I had to install seemed to prevent me from compiling
earlier kernels. So I spent a lot of time backing out 2.4.17 and its dependencies, not to mention fsck'ing my disk a million
times from 17's kernel panicked sloppy restarts. Grrr.
- 1-15
- Still toiling fruitlessly on readin' the 'Pod. Several folks have written me to offer help. The only thing I haven't tried is upgrading my kernel, so I'm trying 2.4.17.
- Grant wrote a firmware updater and needs help porting it to Linux.
Strategy
Use SBP2 (SCSI on IEEE1394) to mount iPod as a disk, read/write using a fs driver for HFS+ or stand-alone utilities, and create the appropriate entries in the iTunes database file to enable new MP3s to be added. This page focuses on details of each task in this approach.
Joe wrote in to suggest another option: emulating the "sync" protocol used by iTunes.
I always asssumed that iTunes just goes in the front door: mounts the drive, copies files over, and updates iTunesDB. But maybe there's an easier
way, a shortcut whereby iTunes sends commands over the Firewire bus like: here's a new MP3, add it to your collection, and add it to playlist 1.
If it was possible to provide simple commands to iPod and delegate all internal housekeeping, we could forego some of the more arduous tasks I've suggested here.
Recognizing the Device
- When iPod is plugged into a Mac, it receives power from the Firewire bus and enters a "hard disk" mode
(3-pointed icon on the screen). But with the PCMCIA Firewire card I'm using, the bus is not powered by default
(requires external power adapter). To force iPod into "hard disk" mode, hold down Menu and the
Play button until the Apple logo shows up, then hold down the |<< (previous) and >>| (next) keys.
- With iPod connected a boot time and Firewire drivers properly installed, I see the following message at boot time:
ohci1394: v0.51 08/08/01 Ben Collins
PCI: Setting latency timer of device 02:00.0 to 64
ohci1394_0: OHCI-1394 1.0 (PCI): IRQ=[11] MMIO=[20800000-20800800] Max Packet=[2048]
ohci1394_0: Physical register received outside of bus reset sequence
ieee1394: Local host added: node 1:1023, GUID 00:c0:d0:00:00:f8:65:57
ieee1394: Device added: node 1:1023, GUID 00:0a:27:00:02:00:11:17
insmod raw1394 and use gscanbus to see an "Unknown" firewire device connected.
insmod scsi_mod and insmod sbp2 to add iPod as a Generic SCSI device (/dev/sg0).
SCSI subsystem driver Revision: 1.00
ieee1394: sbp2: Driver forced to serialize I/O (serialize_io = 1)
scsi0 : IEEE-1394 SBP-2 protocol driver
ieee1394: sbp2: SBP-2 device max speed S400 and payload 2KB
scsi: unknown type 24
Vendor: GHIJKLMN Model: OPQRSTUVWXYZ ~zz Rev:
Type: Unknown ANSI SCSI revision: 03
resize_dma_pool: unknown device type 24
- Installed the Generic SCSI Driver utilities and was able to get data (using sg_dd). Note some recognizable strings like
(C) 2001, Apple_HFS, and iPod.
- Even if iPod uses HFS, SBP2 surfaces it as a generic SCSI device which is unmountable (not a block device).
But thanks to Ted Whalen, I was able to recognize iPod's partitioning scheme by enabling kernel option CONFIG_MAC_PARTITION.
- Ashkan advises that mounting iPod is much easier if you use
modprobe sbp2 serialize_io=0. To see that iPod is on the bus, he suggests using cdrecord --scanbus, which returns this comforting message:
scsibus1:
1,0,0 100) 'Apple ' 'iPod ' '1.00' Removable Disk
- Mark has been trying to help me with my sbp2 problems. He says:
Theres a bit of code in drivers/ieee1394/sbp2.c that looks like:
/*
* Check for Simple Direct Access Device and change it to TYPE_DISK
*/
if ((scsi_buf[0] & 0x1f) == TYPE_SDAD) {
SBP2_DEBUG("Changing TYPE_SDAD to TYPE_DISK");
scsi_buf[0] &= 0xe0;
}
I added a similar check right after it:
/*
* MCG: Check for iPod
*/
if ((scsi_buf[0] & 0x1f) == 0x10) {
SBP2_DEBUG("Kludging iPod to TYPE_DISK");
scsi_buf[0] &= 0xe0;
}
- I tried the above hack to sbp2, except I altered it slightly. I checked for device type 19 and 24, since the value seems to oscillate between those two.
My system log proves that I'm indeed kludging to TYPE_DISK, but I still haven't been able to read the iPod.
If I try to mount it using hfsplusutils, I get
(null): /dev/sda3: Neither Wrapper nor native HFS+ volume header found (Unknown error 4294967295).
Using the HFS+ kernel module isn't much better: HFS+-fs: unable to find HFS+ superblock in the system log.
I'm still getting the resize_dma_pool: unknown device type errors in my log (which are apparently coming from the scsi side of things).
So I'm still stuck on this step. Several folks have far surpassed my progress here, most recently Martin who is using Debian/Woody, hfsplusutils, and Mark's sbp2 hack and can copy files off of iPod.
Accessing the Filesystem
- iPod uses HFS+ which is NOT a supported filesystem in Linux.
- HFS+ utilities are not baked and currently only support reads, not writes. Even
worse, I can't get hfsplusutil to mount my iPod at all, although Ted has had success on LinuxPPC,
including copying MP3s off of the device. Here is a transcript of his iPod hacking.
- Another approach was suggested by Racter: reformat iPod using FAT16, which he claims the iPod OS supports.
- I have yet to hear of any successes in trying this route. Even if iPod will recognize FAT16 partitions, it will limit the device to 2 gigs.
- Word of caution to would-be partition warriors: iPod Hacks has a story from an individual who reformatted their iPod as HFS (instead of HFS+). They found that
iPod refused to recognize the drive and were forced to mail it back to Apple for reinitializing.
- Antonin is working on extending hfsplusutils to support write operations. Initially he's testing his ideas in Python. Note that these scripts are experimental and could damage your iPod's filesystem.
- copy.py copies a local file onto iPod. Requires hfs_volumes.
- Configuration: Modify the first line of copy.py (dev_path='xxxx') to point to the third partition of your iPod.
- Usage:
python copy.py [file in current directory] [destination path on iPod]
- Example:
python copy.py test.mp3 iPod_Control/Music/F10
- An alpha version of a Linux HFS+ file system driver is available at SourceForge.
It's read-only for now, but still great news.
Ira suggests that you may need this patch in order to get the driver working.
I've sorta given up on hfsplusutils and have started trying to get this driver working so I can poke around on the iPod's file system.
- Peter found HFS+ source for Darwin (requires Apple Public Source License account).
Updating the iTunes Database
- MP3s will be ignored on iPod without being referenced in iTunesDB, a proprietary, binary format file.
Playlists are also stored in this file.
- Stewart graciously provided his iMac to help the iPod hacking cause. We examined its file system using OS X's shell.
Here are some of the more interesting files:
- Ted has written a Python script to parse the iTunesDB file format.
He also provided this screenshot of a hex dump of iPod's firmware.
- Firewired has been able to get his iPod working with Windows by creating his own iTunesDB.
He modified iTunes DB with a hex editor and can get iPod to pick up any songs he wants, no additional files or black magic necessary.
He offers this iTunesDB as a sample of his hex editing wizardry. It contains pointers to 16 MP3s. All must reside in ipod_control/music/F00 folder and be named 01.mp3, 02.mp3, and so forth. This static naming convention of MP3s seems like a clever way to avoid constant tinkering with iTunesDB!
- There is a catch to the above approach. I just read on iPod Hacks that the song length is encoded in the iTunesDB.
The sample file above sets all songs to a duration of 5:52. If the song is shorter, the playback is fine. If it's longer, the song is cut off at 5:52.
- Firewired provided a new iTunesDB, this time with 80 songs, split into playlists of 20 songs a piece.
Not that you don't need all 80 songs to use this DB. Each song is tagged as being 4 hours and 40 minutes, which is more than enough to allow most songs to complete,
but also makes iPod's song duration meter less than useful.
- Antonin, who is working on a Python script to update the iTunesDB, provides this script to dump the contents of an iTunesDB, as well as this analysis of the iTunesDB file structure:
- The file is composed of records. Each record has a header and a data part.
- The data part can be empty (0 bytes), an Unicode string, other records, or unused space.
- The header has a 4-byte identifier, followed by the record's header length and various information based on record type.
- Record types and hierarchy:
MHDB : database record
MHSD : list holder
MHLT : song list header
MHIT : song item
MHOD : song item strings
MHLP : play list header
MHYP : play list item
first MHOD is play list name
second is empty space ????
and after comes MHIP/MHOD pairs
MHIP : index to song item (MHIT record)
MHOD : empty space ?
- Analysis of MHIT (song item) record:
0000e1a0: 6d68 6974 9c00 0000 7202 0000 0600 0000 mhit....r.......
0000e1b0: 2408 0000 0100 0000 0000 0000 0101 0000 $...............
0000e1c0: 7ede 3eb8 500d 2500 ebfe 0100 0c00 0000 ~.>.P.%.........
0000e1d0: 0d00 0000 d107 0000 9400 0000 0000 44ac ..............D.
0000e1e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0000e1f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0000e200: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0000e210: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0000e220: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0000e230: 0000 0000 0000 0000 0000 0000 6d68 6f64 ............mhod
mhit : record type
9c00 0000 : record header size
7202 0000 : record size (header + mhod records inside)
0600 0000 : number of mhod records inside
2408 0000 : record index
0100 0000 : ?
0000 0000 : ?
0101 0000 : file type 0x101=>VBR, 0x100=>256 kbit/s, other...
7ede 3eb8 : file date in apple format (number of sec since
01/01/1904)
500d 2500 : file size (in bytes)
ebfe 0100 : song duration (in 1/1000 of seconds)
0c00 0000 : song order in album
- Grant is working on a WinAmp plug-in to manipulate iTunesDB. So far he has developed a C++ program that will parse and dump the contents of your iTunesDB.
Brad produced a Unix version.
- Check out Grant's website: http://www.podtronics.com.
He wrote a firmware updater for the iPod and needs help porting and testing it on Linux.
This source code is current as of 1-15-02, but check his website for the latest.
- Joe also wrote a Windows program to manipulate iTunesDB, but this one can actually update it with new data.
His website is at http://www.mentaljewelry.org/ephpod. It's written in Delphi 5 (Pascal),
and he has offered the relevant source code if anyone is interested. Also, he learned several interesting things in working with iTunesDB:
- The first playlist must be a playlist of every song on the system that
you want to show up on the iPod. The name of that playlist is the "name"
of the iPod (it will show up on top of the thing while it plays individual
songs and whatnot).
This way you can have "hidden" songs (i.e. only visible in playlists) if
you leave them off your main list.
- iPod will choke on playlists that begin with non-alphanumeric characters, or those that reference
songs that don't appear in the songs portion of the database. In these situations, playlists above the first "bad" one will show up,
the name of the invalid one will be blank, and subsequent playlists will be ignored.
- FonkiE wrote a Perl script to parse the contents of iTunesDB,
and provided a sample dump of its execution. He has another script to parse the EQ settings file (sample dump).
He requests that someone send him an iTunesDB in which each field is self-described: e.g., title contains "Title", artist "Artist", etc. and all of the fields are completed (including filename, genre, size, bitrate, vbr).
- Michael simulates the iPod drive using a Zip drive so that he can play with EphPod and other tools without
the hassle of rebuilding his real iPod when things go wrong.
He formatted a Zip disk as HFS+ using MacOpener and built the directory structure described on the EphPod web page.
Links
Last updated: Sat Jul 3 09:43 am EST 2002
