Actually the difference is also in the red. On a dwarf:
Red over Yellow: Restricting
Yellow over Yellow: Approach Slow
Yellow over steady Red: Slow Approach
Yellow of flashing Red: Medium Approach
If you look in the NORAC book you will see all the possible ways each signal can be displayed depending on where you are.
For example coming in on a Franklin train you may see a Sloq Approach three different ways:
Walpole West: Red over Red over Flashing Yellow
Cove (just east of Back Bay): Yellow over steady Red
Tower One: White domino signal: positoned with white on bottom left and white towards top right.