I assume they could have been voted off by the other Board members. I haven't seen their Charter but I would assume a certain number of votes will remove a Director.
HBG held a 4-person voting block on the 7-person WT Board:
Tony Andreacchio
Julie Romero
Denny Burgess
Rick Wayde
It was previously a 5-person block when Simon Cook was on the Board and Rick Wayde was officially the Wests Magpies appointee. Rick Wayde managed to be Magpies rep because HBG also hold (still) a voting block at the Magpies level - including how they removed Shannon Cavanagh.
I am not clear if John Dorahy became the new Wests Magpies rep when Simon Cook stepped down; either way HBG can control the WT Board as a block.
The independent members were Lee H and Danny Stapleton, which was obviously somewhat of a farce because Lee was also the major sponsor and Danny is Balmain down to his bones.
So if any of the HBG members refused to step down, the other members should have been able to vote to remove them. The controversy/trouble would have been that HBG Directors would have to vote against their own people, and Lee H could have teamed up with sufficient disenfranchised HBG Directors to block the move. I am going to assume that HBG unanimously agreed to vote to remove themselves and therefore any other votes were moot.
Also to note now, Danny Stapleton has been reappointed as a new Director, officially now as Balmain rep rather than independent, and HBG's new appointee is Dave Gilbert who replaced Mike Bailey when he died. Magpies haven't appointed their rep yet - it remains to be seen whether HBG will put another of their own people on via their Magpies control, or maintain face and appoint an entirely new person.