I give preference to local art stores, and am very fond of the one nearest my house - Muse Art & Design on SE Hawthorne in Portland, Oregon, which carries a lot of unusual products like dip pen nibs for cartoonists and Peerless liquid watercolors. However, there are many products you can't buy at Muse or anywhere else in town for that matter. Noodler's Inks are conspicuously absent (though Muse is hoping to carry them eventually), and also many Japanese products and many fountain pen brands. As a result, despite being a frequent visitor to Muse and other local stores, I have also ordered from several online pen retailers and generally they've been fantastic. For those of you that are new to ordering pens and pen related things online, here are my experiences.
Note: I don't want to start any fights here, and will probably not approve any comments that seem argumentative regarding these retailers. I'm sure there are other great suppliers out there that I haven't heard of, too, so I don't mean to insult anyone by not including them. These are just my own experiences to be taken as nothing more than that - end of story.
Goulet Pens is stellar. They're a small business with a big footprint in the online pen & ink community. They offer a lot of excellent inks, ink samples, a monthly ink sample club, Noodler's pens (when they are available for restocking), and many glamorous expensive fountain pens. They're operating their business in a very modern way, interacting with their customers quite actively all over the internet, providing streaming video covering new products, commenting on other site's reviews, and blogging with their own experiments. Their customers tend to call them by their first names. Orders go out pretty much instantaneously. All around perfect retailer. (No affiliation, except as a repeat customer.)
Writer's Bloc is based here in Portland, Oregon, but has no physical storefront. They offer the Exacompta Sketchbook that I'm using now for a great price, a lot of the more affordable fountain pens, and good inks. Their website is not perfect - why do you have to go through the "writing instruments" category to find inks? - but it works. I ordered one item from them on a Friday afternoon, it was shipped on Monday, arrived on Tuesday, so local was totally an advantage. Good prices. I don't know much about them but my one order with them went very smoothly & quickly. (No affiliation.)
JetPens probably goes without saying. I love the free shipping for orders over $25. Order turnaround is fast, deals are great, they've got awesome stuff from Japan like very fine gel pens and brush pens, and it's just a hassle free good time. (I have an indirect relationship with them, since they are a sponsor for the Pen Addict which I do reviews for - but this was my opinion about them for long before that relationship existed. I've been buying from them for several years.)
I Sell Pens is another online retailer that offers some hard-to-find pen types such as the Noodler's Flex Pen and the Hero M86 Chinese Calligraphy Pen, as well as many fountain pen inks. I have only made one order from them, but it all happened exactly as hoped. I got my name on a wait list for the Noodler's Flex pen, got a personal email when it was "about" to be back in stock, went online that evening and grabbed one fast. I threw in a fountain pen I'd never heard of because he was offering it for something like $3. The order was shipped immediately. The $3 pen was not wonderful, but for $3 - seriously? - it is a decent pen and made for a fun experiment. Their site isn't very well designed (you do a lot of scrolling through pages looking for the item you want, instead of just searching for it and having the right thing pop up) but they do the job and have some unusual items. (No affiliation.)
I had one bad experience in ordering online, and that was with Swisher Pens. It seems like that business has been going through some flux this year so maybe my experience wasn't representative of their service in the long run, but communication was very poor and I never got my item. They did refund my PayPal payment - in fact, that's how I found out they had canceled my order after keeping me waiting for a month. Ah well. Plenty of other wonderful retailers to choose from. (No affiliation.)
Just my opinions, but I hope this helps direct you to some good, solid online retailers to work with next time you need a fountain pen ink fix or want to acquire that weird Chinese pen you saw someone drawing with that one time.