Summerlicious is on in Toronto right now. There are a large selection of excellent restaurants with fixed price, 3 course menus. It's a great way to check out some of the food culture in the city. You can check out the website here: http://www.toronto.ca/special_events/summerlicious/2013/index.htm
If you want a low-key afternoon, hit up 'The Beaches' community in Toronto. It's on Queen St. East. There are plenty of great little shops to check out, as well as a pretty nice beach, though the water in Lake Ontario isn't great for swimming in.
Once you're done catching some sun, be sure to check out "Ed's Real Scoop" on your way back into the core of the city. It's a very popular spot to grab ice cream. They make all of their ice creams in house and feature a rotating selection. My personal favorite combo so far was a scoop of Dark Chocolate and another of Stout ice cream.
If you want some late night low key fun, hit up the Annex. It's on Bloor St. West. There you'll find Honest Ed's, BMV and Future Bakery, which features a ton of cakes that you can order by the slice. If you've got a group together you can also hit up Snakes & Latte's, a sit down bistro where you can play a nearly endless variety of board games with friends for as long as you like for the cover charge of $5. Food & drinks are extra, but they'll deliver it right to your table.
Queen West used to be the off-beat neighborhood, but it's been overrun by the mainstream sprawl. Still, it can be a fun place to wander, grab a Starbucks and do some shopping. There are still lots of pubs and bars on the strip, and the odd independent store that makes it worth checking out.
If you want to mingle with the upper crust, hit up Yorkville. It's the area just north of Bloor St. between Spadina and Yonge St. Lots of boutiques, couture and designer clothing shops, over-priced coffee spots and some very calm green spaces.
Of course Gaytown aka "The Ghetto", is on Church St. Primarily north of Carlton, and south of Bloor, it's where most of the homos go. It has become more mainstream over the past few years, and while many gay men in the city have moved outside of it, there are still lots of men who flock there every weekend.
In terms of shopping hit up "Out on the Street" or "Priape" for gay paraphernalia, ie. underwear, lube, tight tshirts, leather, and sex toys. Priape is always fun even just for a laugh. If I remember correctly they still have a horse cock dildo, and some extreme leather & rubber gear.
I'm sort of clueless when it comes to gay clubs as I haven't been to one in months. There has been some shake up as a few clubs have sold their property to make way for more condos.
If you want to workout while in the city, Goodlife is your best bet as they're all over the city and have a near monopoly. For a single visit they'll charge you $10.
Hope that helps! If you have any specific questions let me know.