If you had unlocked the Locust of Void achievement, he will start with that trinket too.
His active item, Void, uses -2 less charges. These items are equally as likely to appear as the Horseman's other items, and, if said Horseman can drop either a Cube of Meat or a Ball of Bandages and the player already has at least one of either, it'll be dropped instead. While not exclusive from them, each Horseman has one item that can be dropped from it instead of their usual rewards. In addition to their usual drops, The Harbringers have unique drops.