I guess I was right - searching does have its worth, even in Akihabara. If you look hard enough, you eventually find this place, Kamifusen (Paper Balloon), in the middle of some faceless computer shops.
This has to be one of the strangest stores I've ever been to. How many other stores do you know sell porno Windows software, pirate multicarts and paintball guns - some of which would get you shot at by cops if you carried them around in LA - under one roof?
Pirate games and multis, although still hard to come by, are much more easily available in Japan than in the US. Prices are high, however. The average multi costs 4000 yen and up, and a really good HK original like King of Fighters or Somari can run 6000 if you can find it. In the end, going to the original manufacturers' lands - Hong Kong and so on - is best.
On the weekends in Akiba, there is a stand put up on the sidewalk (usually a few meters away from MediaLand) featuring nothing but pirate carts and multis. It's easily noticeable by the portable TV with a Famicom attached, showing off some HK fighting game or another. Prices here run an average of 5000 yen, so if there's a SF2 clone you absolutely have to have, grab it here. The guys that run the stand are very cool, and are more than glad to demo a game for you if you ask (although they understandably turned down my request for a photo).