For this to happen, you have to have at least two problems.
1) Fuel petcock is in "prime' position, or the vacuum petcock is not shutting off fuel in "on" or "reserve"
2) Fuel leaks from carb into intake manifold - usually a float issue. While pretty rare, I have seen sunk floats before. When they're sunk, even new float needle/seat won't help. If they're sunk, you can hear fuel sloshing inside.