One of the factors that would impede collectibility for some would be the generic nature of the later models. While older examples come in wide and narrow widths and are every bit as sophisticated a deck in terms of manufacture and design as any of USPC's other well-known brands, at one point USPC simplified the manufacturing process for Congress cards. For one, it's rare to see a non-vintage or late-model vintage deck that isn't narrow width (bridge sized) - for some odd reason, most collectors stay away from narrow decks and they're not anywhere near as popular. (Having shuffled a few decks in my past, I can understand why a man with average-sized hands would find narrow cards a little harder to work with.)
Another issue is that, in order to simplify the printing of Congress decks, USPC often printed large quantities of uncut sheets with just the faces of the cards in the Congress design. They would sit on a shelf, waiting for a back design to be decided upon and created before going to press. These pre-faced sheets were printed without an Ace of Spades code on them, as it was impossible to know in advance exactly when a particular sheet would get its back printed and become a finished deck. As a result, it can be very difficult to date a deck's age, especially in the years following the abolition of the Federal tax on playing cards. As you noted, these late-model Congress decks were frequently sold in pairs without the use of a deck seal - each deck was plastic-wrapped, put in place, then the whole box was plastic-wrapped, meaning no deck seal, stamp or sticker to be found.
That generic nature of the cards turns off many card collectors, who like to collect cards that can be dated and who's rarity is more easily determined. As mentioned, practically every theme under the sun has appeared on the back of a Congress deck at one point or another. A completist collector would go crazy trying to get all the varieties - just getting all the decks made in a given particular year could be challenge enough! It's easier today, as USPC under the Jarden Corporation seems to have streamlined the entire brand - of the most recent catalogs available online from the bicyclecards.com website, the domestic 2013 catalog lists only seven back varieties, all sold as two-deck bridge-sized sets with either standard or jumbo indices, and the 2009 international catalog lists only three of the seven.
I think the one thing that would give Congress collecting a shot in the arm would be the cancellation of the brand! Suddenly, they'd be out of print and thus more scarce over time, especially in new or lightly used condition. Eventually, some Congress decks would rival some of the most scarce Bicycle decks in terms of rarity and collectors' price.
Well, that's my two cents' worth of rambling - anyone else?