By David Kelly
For nearly 80 years now, most Americans have had federal income tax withheld from their wages through a system that has been in effect continuously since 1943. The current withholding legislation was enacted during World War II and was sold as the patriotic choice to help the United States win the war. The war ended with victory in 1945, yet the tax withholding conveniently remains, funding our federal government labyrinth.
Today, American workers know that their gross and take-home wages are two different things. They have accepted payroll withholding as a fact of life in our complex world of government taxation, and hope to avoid possible tax penalties — or worse — when taxes are due each year.
The Internal Revenue Service (IRS), which collects these taxes, has recently been making headlines with the passage of the so-called Inflation Reduction Act and the hiring of 87,000 new agents. Those agents are going to be looking to ensure the federal government gets its “fair share” of your hard-earned income, as some Americans have found ways to avoid paying their taxes in full.
By design, withholding of federal taxes reduces tax evasion by seizing the people’s earnings before they can possess them. The system of mandatory federal tax withholding at the source of income is a brilliant way for our government to gain its operating funds. It is nothing short of legalized thievery, as involuntary taxation is a form of theft. More than that, a federal income tax with mandatory withholding essentially enlists employers as unwitting agents of the U.S. government. As Dan Smoot, a former FBI agent and author of The Dan Smoot Report, wrote in the May 7, 1962 issue of his report,
The role of unpaid tax-collector, which the withholding law thrusts on employers, is a heavy burden, and has destroyed many small and struggling enterprises. Businesses with enough operating capital can survive the additional burden, by passing it on to consumers in higher prices; but this is one of many governmental harassments which have so inflated the cost of doing business in the United States that American-made goods are being priced out of world markets.
This was written in 1962; consider how much worse the situation has become since then.
But how did we get to where we are now, with the mandatory withholding of federal income taxes?
Before federal income taxes were ever collected, the federal government’s largest source of revenue came from tariffs that were collected at major ports. This was a simple way to fund the federal government early on, as the U.S. government’s responsibilities were limited to basic operational matters and did not include such modern expenses as social-insurance programs, welfare programs, agricultural subsidies, and more.
Another source of revenue was (and still is) federal excise taxes applied to specific items such as motor fuels, tires, telephone usage, tobacco products, and alcoholic beverages.
In 1791, during the presidency of George Washington, Treasury Secretary Alexander Hamilton proposed a tax on distilled spirits as a way to pay off some of the debt from the War for Independence. After heated debate, the House decided by a vote of 35–21 to approve legislation imposing a seven-cent-per-gallon excise tax on whiskey. This became the first time in American history that Congress voted to tax an American product, and it led to the Whiskey Rebellion. Washington quelled the “tax rebellion” in 1794, suppressing the insurgency with 13,000 militiamen provided by the governors of Virginia, Maryland, New Jersey, and Pennsylvania. Thereafter, the whiskey excise tax remained difficult to collect, and it was repealed in 1802 during Thomas Jefferson’s presidency, as he, along with many other Americans, opposed Hamilton’s Federalist tax policies.
Later, during the War Between the States, the Union government needed money to fund the war effort. A federal income tax, along with withholding, was instituted and collected with the passage of the Revenue Act of 1861. The U.S. Treasury withheld taxes owed by federal employees under this income-tax law until an 1864 amendment exempted federal salaries from taxation. Although withholding taxes ceased, the federal government continued to collect a progressive tax from citizens at three percent on incomes between $600 and $10,000 and five percent on incomes over $10,000. The war-spawned income-tax law was repealed in 1872.
No income tax was collected from 1873 to 1893. In late 1894, the Revenue Act, or Wilson-Gorman Tariff Act, was passed. This slightly reduced the U.S. tariff rates from the numbers set in the 1890 McKinley tariff and imposed a two-percent tax on income over $4,000, and no withholding of earnings was implemented. That income tax affected only the rich, and it was estimated that less than one percent of Americans households would pay any tax.
In 1895, the U.S. Supreme Court declared the federal income tax unconstitutional and struck down portions of the Wilson-Gorman Tariff Act of 1894, as it imposed a direct tax on the incomes of American citizens and corporations. Any direct tax must be subject to the rules articulated in Article I, Section 2, of the U.S. Constitution: “Representatives and direct Taxes shall be apportioned among the several States which may be included within this Union, according to their respective Numbers.” There were no federal income taxes from 1895 to 1912.
With a direct federal income tax deemed unconstitutional, Congress sought simply to amend the Constitution. By adding the language, “from whatever source derived,” the “direct tax dilemma” related to Article I, Section 2 would be resolved, and Congress could be authorized to lay and collect income taxes without regard to the rules of Article I, Section 9 regarding census and enumeration. The 16th Amendment was ratified in 1913.
The full text of the 16th Amendment is: “The Congress shall have power to lay and collect taxes on incomes, from whatever source derived, without apportionment among the several States, and without regard to any census or enumeration.”
With the Tariff Act of 1913 enacted after the ratification of the 16th Amendment, tax rates on Americans ranged from one percent to seven percent based on income in 1914, and grew to a progressive rate of up to 77 percent of income by 1917. The tax was withheld at the source, or by the employer. This system provoked so much complaint from employers that even the secretary of the treasury, William Gibbs McAdoo, recommended its elimination, and in 1917 Congress withdrew its authorization for withholding taxes.
It wasn’t until the passage of the Social Security Act in 1935 that payroll taxes were once again withheld by employers, but that didn’t include income taxes. Individuals still paid federal income taxes owed during a given year over the following year in quarterly installments. And back then, relatively few people paid income taxes. As late as 1939, fewer than seven million individual returns were filed, and the filers’ total tax bill came to less than $1 billion, or less than four percent of their net taxable income. Since so few people paid income tax, and the amounts due in most cases were so small, paying them once a year or every quarter imposed no great burden and gave rise to few taxpayer complaints.
With the Great Depression dragging on, and as the government began to mobilize for participation in a world war, its revenue demands started to grow swiftly in 1940. Federal spending burgeoned from $9 billion in fiscal year 1940 to more than $92 billion in fiscal year 1945. Most of this spending was financed by borrowing, but huge increases in tax collections also took place. In 1945, nearly 50 million individual income-tax returns were filed, and the filers owed more than $19 billion, or almost 20 times the amount that Americans had paid just five years earlier.
In the dark days of World War II, the nation was basically bankrupt and needed money to fund the war effort, and income taxes were still only collected quarterly. The government needed money now. It was free-market economist Milton Friedman who came up with a solution while he worked in the early years of World War II at the Department of the Treasury in the Division of Tax Research.
“It was clear to all of us at the Treasury, as we set out to multiply the amount of revenue to be collected from the personal income tax, that it would be impossible to do so unless we could develop a system to collect the taxes as the income was earned, not a year later,” Friedman said, according to his 1998 memoirs Two Lucky People, written with his wife, Rose.
Thus, the current pay-as-you-go withholding system was the solution. The problem then was how to switch to “pay-as-you-go” so that when the switch was made, the taxpayers would pay two years’ taxes in a single year — the amount due under the old system on the previous year’s earnings, as well as the amount due under the new system on the current year’s earnings. Outside of the complaints that this double taxation was sure to elicit, many people could not make all the tax payments, especially when tax rates were also being increased.
Transitioning from the old tax payment plan to the pay-as-you-go withholding plan caused quite a bit of debate in the government and among taxpayers. In 1942, Beardsley Ruml, the treasurer of R.H. Macy & Co. and chairman of the Federal Reserve Bank of New York, proposed to “forgive” the previous year’s tax liability completely when the switch to the pay-as-you-go system was made. But the Treasury objected to allowing the “forgiveness” and proposed an alternative plan.
In 1943, a Senate hearing was held on a new tax payment act. During the hearing, senators and representatives spoke candidly of the revenues that needed “to be fried out of the taxpayers.” The wartime Congress passed the Current Tax Payment Act into law on June 9, 1943. The act provided for a complicated partial-forgiveness transition. As Friedman described it, the law basically “canceled … one year’s tax obligations of $50 or less and 75 percent of the required tax on the lower of 1942 or 1943 income, requiring the remaining 25 percent to be paid in two equal annual installments.” When the system became fully operational, employers withheld almost $8 billion for income taxes in 1944, and more than $10 billion in 1945.
It’s been 77 years since World War II ended, but instead of revising the Tax Payment Act of 1943 to fit the needs of postwar America, the government has kept it in force. This has given the federal government too much power and more than its fair share of the taxpayers’ income.
Charlotte Twight, in her article “Evolution of Federal Income Tax Withholding: The Machinery of Institutional Change,” said it best: “Withholding is the paramount administrative mechanism that since 1943 has enabled the federal government to collect, without significant protest, sufficient private resources to fund a vastly expanded welfare state.”
Friedman, who admitted being “one of the architects” of the Treasury’s proposal for a withholding system, noted in his memoirs that the system “would have been introduced had I been involved or not.” Withholding was an essential element of the government’s wartime revenue grab. “At the time,” concluded Friedman, “we concentrated single-mindedly on promoting the war effort. We gave next to no consideration to any longer-run consequences. It never occurred to me at the time that I was helping to develop machinery that would make possible a government that I would come to criticize severely as too large, too intrusive, too destructive of freedom. Yet, that was precisely what I was doing.”
In 2021, Americans filed more than 261 million income-tax returns. The vast majority had their income withheld by their employers.
Now imagine that the withholding of federal taxes, or, for that matter, of all taxes from wages, was abolished, and the taxpayer could keep all his earnings. Then each taxpayer would choose to file an annual or quarterly tax payment to cover that year’s vast array of taxes due, including federal, state, Social Security, and Medicare. Would that not be the biggest eye-opener as to the extent of government spending for all Americans to see and understand as they write huge checks to the government?
What would it take to repeal the current withholding law? Only a majority of both Houses of Congress and the signature of the president. Abolishing the withholding of taxes would be an act of liberty and resistance. It would amount to a peaceful tax revolution, and hopefully improve honesty and openness in government. But why stop there? The 16th Amendment should also be repealed, completely eliminating the federal income tax — which for much of our nation’s history was unconstitutional. Of course, this would require a massive reduction in the size of the federal government, bringing it back to its constitutional moors, which would lead to a commensurate increase in individual liberty.
After all (and somewhat ironically), it was none other than Woodrow Wilson — under whose watch the Federal Reserve and federal income tax were enacted — who said on May 9, 1912, “Liberty has never come from government. Liberty has always come from the subjects of government. The history of liberty is the history of resistance.”