In the 1870s, a land survey supposedly located the geographical center of the state at a small settlement called Hughes Store. The name of the town was later changed to Center City, as it remains, and this live oak, located in the center of the town, was designated the Center Oak.

When Mills County was created, in 1887, many believed that Center City would be the county seat and a 50-acre townsite was platted around a square reserved for the courthouse. A number of mercantile establishments sprang up, along with a blacksmith shop and several saloons. A hotel was erected. The town developed into a populous commercial center. Regular stage and freighter runs between Brownwood and Gatesville and between Hamilton and Lampasas stopped at Center City to change teams of horses, and to exchange mail, passengers, and goods. According to accounts by early settlers, the Center Oak provided shade for early court sessions, until a courthouse could be built, and the area beneath its spreading branches served as a church and as a temporary school.

When the Santa Fe Railroad came to the county seat at Goldthwaite, ten miles west of Center City, Goldthwaite began to grow and attracted Center City's business and townspeople. In the early 1930s, when State Highway 7 between Goldthwaite and Gatesville was to be widened, construction called for removing the old oak. But local sentiment had its way, and the highway department relented and moved the roadway to the north to spare the tree.

Today, all that remains of Center City is a small combination general store and service station, a historic lodge building, a few scattered homes, and this memorial, which marks the spot once designated as the center of Texas.

The Center Oak died in 2011. The remains of its trunk is located 9.5 miles east of Goldthwaite, just 100 feet south of US Highway 84, on CR 332.