^If you've got
Bekka and Billy,
Coming Home and
Try Me, I'd say that's a pretty great start. Unfortunately, you may have a tough time finding the rest of his older catalog on CD.
Of course, if you ask nicely some fans may be able to help you out.
Also if you browse my site you'll find MP3s of most of the songs from soundtracks/compilations, singles, some songs he recorded as a child, a few outtakes/demos, as well as a select few songs from each of his various albums.