Oil coming out of side of the engine generally isn't normal or a good thing. Really though, people can't make good guesses about the problem or possible solutions without a better description.
It *sounds* like it could maybe be a weep hole or a rocker arm shaft plug but talking about those possibilities is going to be just conjecture and may actually give the wrong impression -- better or worse -- about the situation.
FWIW, I'm not able to quickly find a photo of a motor for you to mark up.
ETA: Found a photo, but I'm still guessing here. Do you mean the hole on the far right of this photo, centered on the fourth fin visible from the top?
(
source)
I have no clue what that is unless it's a core plug (like a freeze plug). Does it look like it's sealed with a convex disk of tin? How much is it leaking?
If it is a core plug and it's failing mainly due to age, the proper thing to do would be to remove both / all of them , clean the holes and install new ones. Still, I dunno what that hole would be.