You can refer to the people in a society as the general public , especially when you are contrasting people in general with a small group.
These charities depend on the compassionate feelings and generosity of the general public...
Unemployment is 10 percent among the general public and about 40 percent among North African immigrants.
N-SING-COLL : the N