WATERTOWN — Responding to critics of a passenger rail line between Madison and Milwaukee — including the leading Republican candidates for governor who are vowing to stop the project — U.S. Transportation Secretary Ray LaHood said Thursday that work is going forward.

"High-speed rail is coming to Wisconsin — there's no stopping it," LaHood said in announcing the second installment of an $810 million federal grant for the project.

Milwaukee County Executive Scott Walker and former U.S. Rep. Mark Neumann, who are seeking the GOP nomination for governor, have called the project a boondoggle and waste of taxpayer money.

Both have said they would halt the project if elected, despite the likelihood that by the time the new governor takes office, the state will have already spent between $60 million and $100 million on contracts, designs, locomotives and rail cars. That does not include money spent by the four cities along the route on train stations.

Although the federal government has said the stimulus money can only be spent on rail projects, Walker said he would push Congress to let the state use the money for roads and bridges instead.

"Our state is facing a $2.5 billion hole in next year's budget and a list a mile long of transportation needs," he said Thursday.

Neumann said the project would stick state taxpayers with the bill for maintaining and operating a train system that he said "is not self-sustainable,"

LaHood compared the project to the Interstate Highway System developed during the Eisenhower administration, which he said began "as bold, daring ideas" even though "we didn't really know where all the money was coming from." He pointed out that the program, which cost $130 billion (not adjusted for inflation) survived several changes in political leadership.

LaHood said President Barack Obama is committed to creating a national intercity high-speed rail system. The administration awarded $8 billion in January, including $823 million for Wisconsin.

An additional $2.5 billion will be awarded for high-speed rail projects by the end of the year. Obama has asked for $1 billion to be budgeted next year.

"We need to get infrastructure in place," LaHood said. "Then we'll have the discussion about how the operating part of it will work."

Gov. Jim Doyle said the project would create more than 5,500 construction and engineering jobs over the next three years. He said it's "unthinkable" that a governor would tell those workers to "take off your hard hats and go home."

"This is happening," Doyle said. "Communities that didn't get themselves on the interstate highway are regretting that decision in a big way."

The $46.5 million federal payment was signed Thursday by LaHood and Doyle in Watertown, one of the stops along the proposed rail line. Other stops are planned in Oconomowoc and Brookfield.

The funding will be used to hire engineering firm HNTB of Kansas City, Mo., for the preliminary engineering and final design of the track segments. A $5.7 million grant signed in June paid for environmental assessments of stations in Madison and Watertown.

Earlier this month, Doyle announced the Madison train station would be located at the state's Department of Administration building at 101 E. Wilson St. Work is expected to begin on upgrading the tracks later this year.