Turned out to be the bulbs. Interestingly enough, I noticed that the filament on one bulb didn't look right, but the other one was fine (tested with multimeter), even though neither one of them worked. If I remeber the wiring correctly, they are connected in parallel so if one burns out another one should still work. But in any case, all is well now and the BackOff light is working as expected.