This term originated as a reference to TheMoteInGodsEye, a book.
It did? Not seen that. It does make sense (the aliens in said book having three hands, and make a joke about 'on the other hand' to try and lighten tension, somewhere near the end)
It often implies a sort of either-or, but logic. Hmm, an example is in order:
Well, we could have fried eggs. On the other hand, we could scramble them. But then again, on the gripping hand, that 'click' was the electricity meter running out, and I've got no 50 pences left.
Darn. You'll have to fry them on your CPU [again]. Quick, now, before it cools down! - MoonShadow