HAUL DOWN ONE'S COLORS

v. phr. 1. To pull down a flag, showing you are beaten and want to stop fighting. After a long battle, the pirate captain hauled down his colors. 2. To admit you are beaten; say you want to quit. After losing two sets of tennis, Tom hauled down his color.