Yes, it improves pattern recognition and tactical awareness, and make your moves more precise.
At least in my experience, if i don't solve puzzles for some time, my game decreases in quality considerably.
But, is not only solve the puzzles that get you the benefit. You must understand what is happening there. If you go wrong on the puzzle, analyse it to see the correct responses, and even if you go right but have any doubts, put it to analysis too.
However, the rating gain, when it comes, will be some 20 to 50, at most 100 points, and may be gradual, not immediate. It can of course be more with time, but to expect too much gain can be unrealistic.
No other studying, just puzzles.
Also, say if you played a lot of games as well.
Please elaborate here.