You've got a few options. The most convenient ones would be based off external memory cards like SD or miniSD, as they offer much more space for less money. Older NOR-based flash carts like the Flash2Advance, X-rom, EZF-Advance, etc. are harder to use and are more expensive for the amount of space you get. Your main choices then are the M3 Perfect (not Pro), Supercard (not Rumble), and G6. I've also heard that the EZ-3 and EZ-4 are excellent devices for GBA files, though I don't have any experience with them.
The Supercard is cheaper than the M3 but contains slightly slower RAM. This RAM is what's used to actually run the GBA files, so this means that a few games will slow down or not boot at all. I know you said you weren't interested in commercial games, but it's an important point. It only affects a few games, and there are other patches you can attempt to apply to improve speed, but you should still keep it in mind. The M3 is more expensive but contains faster RAM that should run games without slowdowns. Also, you don't need to patch most GBA games on the M3, while you do need to do so for the Supercard.
For GB/GBC emulation however, any device should function about the same. The Supercard contains a built-in version of Goomba Color Alpha 6, but this is an out-of-date version. On the plus side, all you have to do is copy over *.gb/*.gbc files to your card and launch them through the menu (though saving takes a special button combination and the existence of a *.sav file already there). The M3 doesn't contain emulators built-in, but its PC-side software will combine the emulator cores with the games you copy over with it, creating *.gba files you can launch from the M3 menu. Since the emulator cores are found on your computer, you can easily update Goomba Color to the latest version.
If you don't want to use the built-in versions, you can use the Goomba Frontend to create your own Goomba Color compilation using the updated emulator core. This packages up to 32 MB of games into a single *.gba file. On the Supercard this compilation requires patching to run, dunno about the M3.
It'll be hard to find audio/video software for the GBA. The GBA is only 16 MHz, so it cannot decode MP3 files in real-time. Your two major software options are the GBA GSM Player by Tepples and Music Player Advance. You can also use the M3 software to convert audio files to *.gbs files for usage on the M3 (the Supercard site offers a way to make it work on a Supercard but I only got audio files to work, not video files). I personally recommend GBA GSM Player. Just use a program like Wavepad to convert the files to high-quality WAVs, put them in the folder, and click go.bat. It will produce a *.gba file up to 32 MB in size with up to 150 minutes of music. Quality isn't the greatest, but it's passable. Music Player Advance has the potential to create better sounding files, but I find it to be very cumbersome to use.
For video, the only real software option would be Meteo, which I can link you to if you're interested. It converts video files to *.gba files, but due to the fact that GBA binaries must be under 32 MB, you will get anywhere between 5 and 25 minutes of video per compilation depending on the quality you choose. The M3 again has software you can use to convert to *.gbm movie files, which it can playback, allowing for larger files to be used.
