I would think sports specialist stores would be a better place to buy gym equipment. The staff would have some knowledge of the products they are selling and therefore they could give valid advice. Also sports shops are more likely to offer after-sales service, then again, it depends on the item you're buying. Free-weights, if looked after well, should last a lifetime with little or no maintenance. On the other hand if the item you're after is an elliptical cross trainer, an after-sales guarantee would be essential, for as I have already experienced, they are subject to mechanical breakdowns, requiring part replacement. A good sports or gym equipment specialist should offer this service.
Alternatively, renting equipment, especially an elliptical, may be a good idea if you plan to use it short term, and repairs are often carried out free, subject to the contract. Where I live there is a firm which rents out equipment for up to a year, with the option to buy at a discount.
As for department stores, the saying: Jack of all trades, master of none holds true here. Stick to buying clothes at department stores.