May I suggest that if you are susceptible to paranoia and/or worse, guilt, then this hobby is not for you – it will take away too much of the enjoyment!
I do have to admit, though, that having seen this subject come up from time to time, obstacles such as joint bank accounts and inquisitive (or, worse, suspicious) wives/partners can be difficult to avoid. This has got much worse as we rarely have a need to use cash – particularly of the folding sort.