Sadly at the moment there is no one simple answer to this and frankly perhaps there won’t be for a while.
For the mainstream coins like BTC, ETH, LTC and a few of their forks e.g. ETH Classic, that I’m looking to hold long term, I’ve got a Trezor wallet. This is one of the two well known hardware wallets, the other being a Ledger, that allows completely offline storage and relatively easy access. It has the benefit of giving you complete ownership of your keys, but also significant protection. It obviously isn’t all that agile, though, given that you have to send money to an exchange to buy something new, and incur fees when you do that. In addition, not all coins are supported
The other extreme is to store everything on an exchange. This is frankly a pretty terrible idea unless you’ve not got much and don’t mind losing it. We’re still in an unregulated space here, and frankly exchanges tend to go away sometimes, most often taking a whole bunch of money with them! It’s perhaps not as common as a while back, but there have been instances in 2017.
So is there a half way house? Well yes. These range from online wallets, where you own the keys, to online wallets where you don’t (effectively like most exchanges) and then there are desktop wallets, where you basically sync down the whole blockchain of a coin and own the keys. These are a good alternative to a Hardware wallet, although if your PC is compromised they are pretty easy to remove funds from and there is plenty of malware out there just looking to do that.
Realistically then you need a strategy which is likely different for different coins. I tend to keep a small amount, maybe 0.1 on an Exchange like Bittrex which is relatively reputable, so I can day trade. I then keep long term hold coins on my Trezor where they are supported. Finally, I run a number of desktop wallets for things like NEO, NAV, CLOAK, where I want to benefit from the staking of those coins (a reward for running part of the network). These are currently on a server machine that I use for other things, but I’m likely to move them over to a designated Crypto machine which is never used for downloading or browsing just to minimise risk.