The computer can be in your home or office, but more commonly it is at a romote location.
Some people use the same company that their dial-up account is through to host their web pages.
You do not have to access to the Internet or even have a computer to have a web site, although is useful. As mentioned above, a server is a computer that hosts your web site. The server can be anything from a home PC running a web server to a high speed UNIX based Sun Ultra or Silicon Graphics machine. These computers not only are a home for your web site, they are also are used to deliver the content of your web site to a browser. The company that owns and runs the server can rent out sections of that computer to other people if they want.