Well, I can't say that a wine actually tastes like rocks but I do know that wines definitely reflect minerality. I think using the term "minerality" is a way of trying to specify flavors in the wines that are notable for being grown in rocky soils. Certain wines, especially whites, are famous for expressing mineral. Chablis, Sancerre, and Muscadet are three of the most obvious. In the case of the first two, limestone and clay soils called Kimmeridgian are the factor while for Muscadet it's gneiss and granite that are in the vineyards.
For some people, it's difficult to identify minerality in a wine. The taste can be subtle and usually lingers in the finish. The flavors are savory, non-fruity, and not the toasty, vanilla, or oaky flavors produced in a barrel. In fact a lot of winemakers who raise minerally whites will not use barrels at all because they don't want any of the stronger oak flavors to cover up the minerality. They'll keep the wine in stainless steel tanks until bottling or if they do use barrels, the wood will be neutral. Once you start to notice mineral complexity in a wine, you'll always be able to find it if it's there. For older wines, the minerality and stoniness becomes more intense over time and these characteristics are prized by wine geeks and connoisseurs. Try an aged Chablis or Muscadet and you'll see what I mean.
For us at Windsor, we love these types of wines so you'll always be able to find a good selection in the shop.