Well, Caffeine is a stimulant, therefore it can make hyper and jittery etc, add that to anxiety... I don't think it is a very good idea to be having lots! I would stick to the doctors and medication recommendation.
Cutting back sounds like a good idea.
=]