You guys must all be a lot cleaner than I. I am constantly picking up stuff on my '02 GSF660S that won't come off with Varsol or Iosol (Naptha) solvents.
I just bit the bullet and started cleaning my whole exhaust system with a fine Scotch Brite pad. It is NOT as shiny as it was when it was new, but once you clean the whole thing it all looks the same and still looks shiny. It is good at taking the heat glaze off the headers, with a little work you can clean up the collector and mid pipe and it works well on the muffler can.
Works for me, you just have to make sure you only polish lengthwise so as not to leave any swirl or cross marks.