In England, the Household Cavalry includes musicians who perform both on horseback and by marching. Here is a picture of them on horseback.
Here are their trumpets playing during a pageant performance.
In Belgium, the Royal Mounted Escort includes a Drum and Trumpet Corps. This groups plays to "herald the arrival of the monarchs."
In the country of Oman, the Sultan organizes the Royal Equestrian and Camel Festival. The show in 2006 included 1000 horses, 500 camels, and 1500 musicians. Here is a picture of a group performing.
The following video doesn't have any great footage of a musician on a horse or camel, but it can give us an idea of what the festival is about.
Finally, I saved the best for last. In the past, there was such a thing as a bicycle infantry. Clearly, our military forces no longer use bicycles for travel. In the Netherlands, however, they have maintained the tradition. This includes musicians. Here is a video of the Royal Netherlands Mounted Bicycle Band. :)