The Union lines were collapsing. General John Reynolds' 1st Corps, despite horrible losses, had held for hours but now succumbed to greater numbers, fatigue, and the tenacity of their foe. The Federal's 11th Corps, victims of General Thomas J. "Stonewall" Jackson's flank attack just two months ago at Chancellorsville, again found themselves being driven from the field by the Confederate 2nd Corps, Jackson's last command before his death. As the men in gray surged forward, the Union Army retreated back through the town through which they had marched just hours earlier. Earlier in the day, Major Generals Reynolds and Howard had both recognized the importance of the high ground south of the town. The objective now became holding this ground while awaiting the remaining 5 Federal Army Corps not yet on the field.

In his after action report, 11th Corps Division Commander Brigadier General Adolph von Steinwehr described the futile efforts to hold back the onrushing gray tide. "General Schurz at this crisis sent again for re-enforcements, and I ordered Colonel Coster to advance with his brigade through the town, to report to General Schurz. The Second Brigade I left on the hill, changing their position so as to support all the batteries upon the same, and fill with two regiments the place vacated by the First Brigade. Colonel Coster met General Schurz in town, who ordered him to take a position north and east of Gettysburg, and to check the advance of the enemy, who were pressing for Gettysburg, and before whose overwhelming numbers the First and Third Divisions were forced to fall back. Colonel Coster had a severe engagement with the advancing enemy, but was, of course, not strong enough to restore the battle. He therefore ordered his men also to fall back, and again took up his position on Cemetery Hill, leaving one regiment to occupy the nearest brick houses of the town, which successfully prevented the farther advance of the enemy."

Union Major General Oliver Howard, who had assumed command of the 1st and 11th Corps upon notification of the death of General Reynolds, later summarized this first days events. "The eventful day was over. The First and Eleventh Corps, numbering less than 18,000 men, nobly aided by Buford's division of cavalry, had engaged and held in check nearly double their numbers from 10 in the morning until 7 in the evening. They gave way, it is true, after hard fighting, yet they secured and held the remarkable position which, under the able generalship of the commander of this army, contributed to the grand results of July 2 and 3." Yet this report employed the hindsight which came so clearly in later days. Confederate Lieutenant General Richard Stoddert Ewell, Commanding General of the Army of Northern Virginia's 2nd Corps, would offer, "General Gordon mentions that 300 of the enemy's dead were left on the ground passed over by his brigade. The enemy had entirely abandoned the north end of the town, and Early entering by the York Railroad at the same time that Rodes came in on the Cashtown road, they together captured over 4,000 prisoners." Major General Jubal Anderson Early, commander of one of Ewell's Divisions who contested the ground held by Coster and other 11th Corps troops, stated simply, "A very large number of prisoners were captured in the town, and before reaching it, their number being so great as really to embarrass us."