What's the sweet spot? Would the Masters decks exist if the Rider decks didn't exist? Is it possible for a big company to do things a smaller company can't? Sorry, I don't mean to be argumentative but I don't have answers to these questions.
I think you're correct that there is a niche for some smaller companies out there, but I also think most of the designers are doing this part time. I did some back of the napkin calculations for my micro-business and determined that I could do it full-time with myself and two full time employees (preferably three) but in any event I would have to be a full time administrator (and part time sales guy) to do it. Which meant I would have even less time to do design work than I have right now. In effect, making my hobby my business would kill my hobby. And I'd be taking a big pay cut even if I was wildly successful.
Maybe the niche is for a company that has the marketing skills and connections to take a design from the hobby stage to the middle ground. I know I'd take a smaller profit per deck if a company took a lot of the marketing and fulfillment work out of my hands AND could sell to a larger audience. My designs, of course, are a bad example here, but assume we're talking about someone with a contemporary design. I haven't looked at any numbers to see if anyone can make a profit doing this, though.
Sorry. Wish I had some answers when all I've got are more questions.
Would the Masters decks exist without the existence of the Rider Backs? Of course they would - they'd simply use the same back design as whatever the predominant market design was. They could just as easily have been made with Hoyle Shellbacks or Tally Ho Fan Backs or even Bee Diamond backs. The choice of the Rider Back was simple - it's the most mass-produced deck in America and there are probably more magic tricks, gaffed cards and gaffed decks in that design than any other design in the world. The point of it being made in the first place was to make an improved, more durable version of that most popular design, whichever design it happened to be - in this "timeline", it was the Rider Back. The question has nothing to do with the situation we're talking about.
There are other decks I named that have been in production for some years now and continue to be produced on an ongoing basis. They're not as massively popular as the Bicycle Rider Back deck, but companies are making them at reasonable prices and bulk quantities - probably print runs of at least 20K each.
I suppose the biggest issue about creating such decks is warehousing. A typical one-and-done designer won't have ample space to warehouse unsold decks and time to spend marketing them to retailers and wholesalers as well as to the general public (or at least the card collector community). Even if you had the money to make the deck, it costs more money to store it, and while it remains stored it COSTS you money in the process. Any product that lies around in storage long enough may even be sold below cost just to get rid of it and either reduce your storage requirements or allow you to stock new, fresh products.
When I worked in retail about two lifetimes ago, the company I worked for eliminated all stockrooms for their stores, including the flagship store. They "stored" all the product on the showroom floor, since products in a stock room earn nothing and cost the rent of the square footage. If a customer asked, "Do you have a size (whatever) in back?", the answer is "No, we have no stock room - what you see here is all we have." But two lifetimes ago was also the pre-GUI days of the Internet, back when only the government, military and universities had wide access to it. Companies exist today that have no showroom floor and are thriving businesses. Storage space costs a lot less than a retail store, unless you're talking about some bare-bones place like Home Depot or Costco - which are in essence just giant stockrooms open to the public.
To rein this all back in, there has to be a sweet spot where you haven't printed too many decks that it costs a fortune to store them before they're all sold, but also not so few that the per-deck cost to you is high and you have to pass that on to the customer. Ellusionist and Theory11 and the others didn't just sprout out of the ground fully-grown - they had to be built one product at a time. They're also not the huge companies people seem to think they are - each one employs in the ballpark of a dozen people each, plus or minus a few. That's not much bigger than the micro-company you just described to me. Sure, at first it would be your full-time job, but ownership of a growing company has rewards as well as risks. Grow just enough and you can start delegating some of the heavy lifting while you get to do a bit more of the fun stuff like making deck ideas become reality. It's a matter of whether you're willing to commit to that idea of creating a company instead of just a sideline.
Jackson Robinson appears to be moving in just that direction, and not strictly on the sale of his own products. He created a Kickstarter project for a 20,000-run deck sold at $6 a pack, but unfortunately there's been so much more talk about the limited-edition 2,000-run deck he's giving away with it that it has overshadowed the entire project - with a week left he's less than 40% of the way to his goal. I was really hoping it would succeed, but honestly, I think he would have done better just taking pre-orders for the $6 basic edition and using KS just for the limited edition. He of all the people making decks on KS right now has the greatest potential to become the next full-time deck company.