Even just a cursory reading of the Old Testament demonstrates the importance of sacred space under the old covenant. The patriarchs, for example, frequently commemorated occasions on which they met with God by consecrating the location where they encountered Him (for example, Gen. 12:1–9; 28:10–22). As the Israelites prepared to conquer Canaan, the Lord told them to offer sacrifices to Him only at the place He would choose, which was Mount Zion in Jerusalem (Deut. 12:1–28; 1 Chron. 21–22).
Due to the connection between sacred space and proper, divinely appointed worship, first-century Jews and Samaritans debated the location God chose for people to bring Him sacrifices. The Samaritans believed Mount Gerizim, not Mount Zion, was the place the Lord had chosen. Upon learning that Jesus was a Jewish prophet, the Samaritan woman asked Him to weigh in on the debate. Interestingly, Jesus gave an answer she was likely not expecting—the debate between worship on Zion or Gerizim was about to be rendered obsolete. People would worship not only on Zion or Gerizim but anywhere they approach God in spirit and truth (John 4:21–24).
Jesus was teaching that worship would be offered not only in one place, but that was not all that He was saying. He meant that worship would henceforth be offered in and through Him, for His teaching that worship would no longer be limited to Jerusalem was His implicit claim to have the divine authority to change the worship arrangements that had been in force for centuries. He, in fact, was the Messiah and could set up a new kind of worship (vv. 25–26). Of course, this worship would not be wholly new. God always wanted people to worship Him in spirit and truth—with hearts dedicated to Him and in a manner consistent with His revelation (Lev. 10:1–3; Isa. 29:13–14). But with the advent of the Messiah and His final, effectual atonement for His people, things were necessarily changing. Animal sacrifices would be made obsolete, and one central sanctuary would no longer be necessary. Christ the truth had arrived (John 14:6), and since that day, on into eternity, worship in truth occurs only when we acknowledge Him as Lord and Savior.
None of that means it is inappropriate to have dedicated buildings for worship under the new covenant. What it does mean is that true worship does not occur when people do not approach God in spirit and truth, no matter where the worship is offered.