Caching keeps frequently accessed objects, images and data closer to where you need them, speeding up access to websites you hit often.
Your browser is the first layer of caching, keeping images and data from websites that you visit often. Next the webserver itself has a caching layer, typically implemented by something like memcache, caching information that it would normally fetch from the database on the backend. This avoids the network roundtrip, and also avoids the load and work of running the query to fetch that data again.
Furthermore you can install what’s called a reverse-proxy on the webserver, such as Varnish. This can bring further speedups and performance benefits to your overall architecture.
On the database server you also do a lot of caching. With MySQL you may configure the query cache, which caches query result sets inside of MySQL, eliminating the need to rerun those queries on subsequent calls. And further the database server has various other caches such as the InnoDB buffer cache, to keep blocks of data in memory, reducing slower requests from disk.