St. Petersburg Paradox
I offer you a game in which you sequentially roll a die until you lose by rolling either a 1, 2, 3, or 4. If you've rolled the die N times before you lose, I will then pay you 2N-1 dollars. If you play this game many times, it turns out that your Average Winnings will be 2 dollars.
I now alter the game a bit, stating that you lose only if you roll a 1, 2, or 3. You no longer a lose by rolling a 4. What do you think your Average Winnings will now be?