As with most things with these people..........the truth turns out to be exceedingly complicated......

The U.S. federal government's fiscal year begins on 1 October of the previous calendar year and ends on 30 September of the year with which it is numbered. Prior to 1976, the fiscal year began on 1 July and ended on 30 June. The Congressional Budget and Impoundment Control Act of 1974 stipulated the change to allow Congress more time to arrive at a budget each year, and provided for what is known as the "transitional quarter" from 1 July 1976 to 30 September 1976. (Wikipedia).

Also:

The first fiscal year for the U.S. Government started Jan. 1, 1789. Congress changed the beginning of the fiscal year from Jan. 1 to Jul. 1 in 1842, and finally from Jul. 1 to Oct. 1 in 1977 where it remains today. (src:http://www.treasurydirect.gov/govt/reports/pd/histdebt/histdebt_histo5.htm)

The Federal fiscal year gives elected Congressmen, who begin office in January, time to participate in the budget process for the next fiscal year. In other words, if the fiscal year was the calendar year, the country would run for an entire year on a budget voted on by PREVIOUS Congress - which is precisely what we are edging towards anyway given the 1976 2 month shift and your note about budget "tardiness" of late.