There is also a related thread 'Guys with Macbooks and iphones'
Here are my thoughts to try and maintain privacy:
Use Firefox as your punting browser, and in the Preferences section select never remember history.
I would choose Android over iPhone for a punting phone as you have more control, and can also use Firefox as your browser, again in permanent private mode. Turn location off in settings, it will then be unavailable to all apps. If you need to find your location then turn it on long enough to fix where you are then turn it off again.
Assuming you are using punting email and other accounts in your nom de plume(s) make sure that these are not remembered, turn off the intelligent predictive function which remembers words you type frequently
Anyone being nosy needs two pieces of information, your user name and password, keep them both secret from those around you, and make sure there is nothing to link your punting user names with your real id. If the person being nosy isn't even aware that you have an account at a particular website then they are not going to be prompted to try and look further, and even if they suspect they haven't got anything to start on.
Delete texts and emails as soon as they have served their purpose. If you want create a 'little-dark-colured-book' (Autocensor wouldn't allow me to mention the colour)use one of the free cloud accounts, and access the document directly, don't download it.
Consider using one of the Linux distro's as your punting environment. Ubuntu now loads gracefully next to Windows and you end up with a dual boot system. It doesn't require a lot of specialist knowledge to get a basic system working. (OK this one may not be for everybody, but Linux is a lot more user friendly these days)
Use web access for email (Through a private browser)
These precautions wont keep a determined hacker or state sponsored investigator out but they will help ensure that your private life remains private from those around you.