That depends on your ROM version. The original ones do not allow GPRS to stay on, the newer ones do.

If your provider is T-Mobile, you can set up SMS alerts for new mail via their web site. This is what I use, and it works very well.