Normally, if two back-to-back LEDs light at the same time, that indicates that one of the LEDs is backwards. You’ve already checked for that, with the flat sides. Just in case, double check that the “leadframes” — the metal shapes within the plastic lens –are also oriented in the same direction as each other.
The second thing that could be happening is if there is an accidental connection (e.g., a solder bridge) between signal lines LED1 and LED4. This could be at D14, D41, or elsewhere on the board.