Unlike site pages (for example, default.aspx), a custom application page is deployed once per Web server and cannot be customized on a site-by-site basis. Application pages are based in the virtual _layouts directory. In addition, they are compiled into a single assembly DLL.
A good example of an Application Page is the default Site Settings page: every site has one, and it's not customizable on a per site basis (although the contents can be different for sites).
With application pages, you can also add inline code. With site pages, you cannot add inline code.