No, you should work out with both hands equally.
With free weights, stick to dumbbells for the time being. With machines, use the ones that allow you to work each side individually, like cables. The reason for this is that your stronger side will always compensate your weaker side whenever both sides are required to push/pull the same object (like barbells).
When you are pushing/pulling at the same time with both sides, make sure to only allow the right side go as far as the left. If you're lifting with one side at a time, start with the left, then do the same amount of weight and reps with the right, no more. Push your left side hard, but not too hard so that your body will overcompensate, it'll throw you off because your body won't overcompensate with the right. To be on the safe side, whenever you can, watch your body carefully in the mirror while you're lifting. Make sure your body isn't doing different things between either side, such as your left shoulder going up when doing bicep curls (body is overcompensating) while the right isn't.
This method works to balance the left and right sides of your body because both sides will get the same workload and will eventually even out. Also works for legs.