Our modern-day well-being is built on both private- and public-sector institutions. We need them to be cost-effective, not inexpensive.
The population of the United States was less than four million when our Constitution was adopted. There was nothing remotely resembling a modern-day national government. Nor did the fledgling governments of the 13 states bear any resemblance to contemporary state government, or even to contemporary local government.
The world hasn't just changed since then; the modern world would be wholly unrecognizable to the Americans of the late 18th century. Government wasn't just smaller at our country's founding; the differences between then and now are also differences in kind. Hardly a shred of the modern-day public sector was in evidence when our country was founded.
Consider just a few truths about public goods and services as we know them. The founders gave these truths no thought whatsoever, because they had not been conceived of.