Well, that is probably the right way to think about it. That was probably the way it felt for most of us in the beginning. But, things never stand still, and always move forward. Coins can appreciate in value with time. Prudent purchases of rare coins can turn out to be wise investments. Knowledge is power, and all that.
Looking back to my own numismatic awakenings, I now see that period of time as very different for collectors from the present. It was the early 1980's. Rare coins were available, but you had to physically "go and hunt" for them. There were coin shops and coin shows, but there was no internet marketplace. The only grading service was ANACS, and they were issuing certificates (not using encapsulation). Grading standards were "fluid" (to say the least).
Fast forward a few years, and the rare coin market began to flourish, with confidence injected by PCGS and other 3rd-party graders. Coin grading accuracy became a paramount concern. The value of Grade vs. Rarity was debated. The term "Condition Rarity" was invented. On-line price guides, together with a plethora of specialized reference books resulted in an explosion of available information. Knowledge was still power.
The coin market today is both wide-spread (thanks to national auction companies and the internet), and highly specialized (Early copper, Morgan Dollars, FBL Franklins, FSB Mercuries, etc.). We have come a long way (I will leave it to others to debate whether we have made progress, or have moved backwards - that is a moot point, as we are here now).
In spite of all this apparent progress (eg. change), I think there are existential threats to the coin market that are too big to be ignored. Here (in random order), are some of the ominous trends I see just ahead:
1. Coin use is in decline. Demand for circulating U.S. coins is in a long-term downward trend. This trend is being driven not just by the increasing cost of making the coins, but by the increasing use of debit cards (and other technology) as an alternative to cash. You may argue that this trend is not relevant to collectible coins, which are rare & desirable objects in their own right. There is some truth to this. However, this argument fails to address the longer term issue: who will want to collect coins when there is no one alive with direct experience with their use in daily life?
2. Coin collectors are aging. I do not have any handy statistic to quote regarding the median age of the U.S. coin collector population. However, I have anecdotal evidence that tells me two things: One - the collector base is largely male. Two - the collector base is aging. The problem for the coin market is compound: not only are the current owners of rare coins reaching the age where they wish to sell their collections, but the size of the collector base to absorb those coins is shrinking. Basic economics (supply vs. demand) can readily predict the outcome of such a scenario: lower prices for rare coins!
3. Grading standards are not fixed. I realize it is heresy to say that grading standards have changed (after all, we call them "standards" don't we?). But, let us not shrink from the fact that the third-party grading services (TPG) practice "market grading" vs. "technical grading". Technical grading standards should remain the same. But there are Bull markets for coins (when prices are increasing), and there are Bear markets (when prices stay flat or decrease). It is commonly accepted that market grading standards become more liberal during bull markets and more conservative during bears. Yesterday's MS64 is today's MS63, and the old VF20 is now F15, etc. Coins that are graded & encapsulated in TPG holders during Bull markets still carry that optimistic grade, but will not command the same market price during a Bear market. The result is typically lower bid prices. The lower bids are duly noted and reported by the price guides, and the downward spiral of prices is amplified. Naturally, the converse of this process can also occur when the market cycle turns from bust-to-boom again. However, savvy dealers and collectors know how to remedy that situation - they simply "crack out" all the conservatively graded coins and resubmit them for higher Bull market grades!
4. Costs keep rising. Table rent at coin shows is higher. Airline tickets and hotel rooms also get more expensive over time. The coin dealer is now being squeezed. Costs are heading higher, but the prices collectors are willing to pay for the products do not seem to be moving up at the same rate. The only answer for the dealer is to seek more on-line business, and limit travel to coin shows. This will invariably lead to fewer coin shows, and concentration of activity at the few "premier" shows that survive.
Conclusion: I apologize if this post seems overly pessimistic. I still enjoy collecting coins, and I consider it a terrific hobby and pastime. I wonder how many people in the U.S. will know what purpose coins serve in 2066?