I think that practice reading is the best way. Before computers came along, some books had intermittent diagrams which was very helpful.
Any tips to improve reading of algebraic notation?

Practice and experience help you improve. Everyone goes through a learning curve for chess notation. You can also try reading or saying the chess moves as you play them online. On chess.com, the moves are displayed on the right of the board. This might help you learn the notation and coordinates a little faster.
One tip of advice I recommend is practicing on chess.com vision trainer. Doing this a few times each day might bring noticeable improvement after a while, but after you get the hang of it, I'd cut it out of your daily routine because then it becomes more about clicking speed than how well you know notation.
In the meantime, you can practice vision trainer with coordinates, with moves, with the white side, with the black side, without notation on the board and so on. Mix it up and play until you are comfortable with it.
Many books that I read stem from an actual game, and it is noted. IE: X vs E. Spain. 1902
Simply type it into your browser. Often very easy to find, and also easy and quick to fine the exact position. You can then follow the book and play out the variations online. Making moves, backtracking to a previous position, or going back to any step is very quick.
Not what you were asking for, but I found this to be very helpful. Miss-types are also easy to spot since the online annotation is correct.

It's not hard to read if you give it a little practice. This is a big upgrade on the old notation where they went 1.K-P4 K-P4 2.KN-B3 ,etc. Use a board and look at the annotations and you will get it in no time. One thing to keep in mind is if two of the same piece (rooks, knights, pawns) can go to the same square, try to explain which one is going to that square. I will show in the pgn below
If more than one piece can move there, the notation adds a row or column to say which one. so Bf7 means only one can go there. Note that bishops are locked to their color, so its very rare to have 2 of the same color, so just looking at the target square's color tells you which one unless you have some really screwy game where someone promoted a pawn to a bishop.
lack of a piece letter (N,K,Q,B,R) means it was a pawn move.
the few odd moves are labeled specially, O-O is kingside castle, O-O-O is queenside (the special labels are 100% unnecessary. Kg1 is a castle if the king is on e1, but they felt it was useful to make it special. EP is also noted but if you just drop the pawn on the right square, its also silly to have the special case. Even this notation is getting old, and back when, I guess it made sense to someone to make all the special case junk.
What you should do depends on what your goals are.
- if you want to play tournaments, you need to be able to write it down fast, and should practice that. Modern tournaments allow boards with the rows and cols labeled.
- if you want to review old games in books etc, plug it into some sort of chess board program. This way you can verify each move since it will create the text for you, and you can play forward/backward after putting the moves in, to examine the moves easily. Also many famous games are already online and you can often copy/paste the notation into such programs and play it out without doing anything but hitting forward/backward.
- if you want to visualize it in notation, you need to do that, playing in your head. Many chess programs let you play blind (hidden pieces, annotation created for moves made) for practice.
whatever else? Just focus your training on your goal, and decide if having the board replay is part of what you want or not. I can write down the moves and quickly make each move from a list, but I can't visualize or play blind at all. Not everyone has that skill, but seems like on the average most GM do.

Sometimes you can find a book in PGN format that has all the notation difficulties handled for you. You can view it as one series of chapters in a program like HIARCS, or even put the individual chapters, which are organized as games, into the chesscom analysis page.
Here are two libraries that I have set up along those lines:
https://www.chess.com/c/2XH97kw5Y
https://www.chess.com/c/2s5T5ZugJ
The first is Capablanca's Chess Fundamentals, and the second is the first of the 500 ECO sections.
Here is the Capablanca. It can be downloaded as a single PGN that you can use right away.

This is not something that comes to you overnight, but, trust me, it will come. Just keep doing what you're doing. Set up your board and try to follow the moves to the best of your ability. It might sound silly, but in early days, putting your finger on the move you are making can help you keep track. Also, starting out with a collection of games that have larger print and more white space is very helpful. I find, even after many years, that I have trouble following the games in Chess Life simply because the print is so small (and I'm 71). Good luck.
Chess notation is like any other language: English, Spanish, Norwegian, python, Morse,...everyone has different aptitude for language. Some people are fluent in 6 languages; others can barely speak their native tongue.
Also, people learn differently. Chess is a highly spatially-oriented endeavor and if you don't think spatially, you will be slower to learn. My suggestion is to find out what kind of learner you are [visual, aural, tactile, imitating others, etc] and to adapt your study.
For example, if you are a tactile [sense of touch] learner, get out a physical chess board and mark the files and ranks and move pieces around and write down the corresponding moves. To me, that's markedly different than watching it on a computer screen.
Elaborating on what @msteen above wrote, what I find difficult are games with many sub-variations; it gets difficult to figure out which line you're in. To keep things simple, choose games with no annotation or alternate lines. You won't get as much out of the game itself [a well-annotated game is worth hundreds of un-annotated ones] but you will get more practice getting comfortable with the language of chess.
A diagram here and there [usually just before the critical move or phase] is helpful so you can validate what you've done so far is correct.
I've been reading some books lately, and I feel like my biggest problem with the books is my difficulty reading them. My biggest problem is with the notation. The book lays down a few moves, and it's too difficult for me to accurately read the notation. I have tried to play the moves out on a board, but very often like 5 moves down the sequence laid down in my books, I would go "wait, hold on, this isn't right", realize I made an inaccurate move at some point, and be forced into starting at the beginning of the sequence. My best way to overcome this thus far has been to plug the moves into an analysis, which gives me reassurance that I am, in fact, following the moves in the book. However, this is a bit time consuming in situations where the book just has a picture of a diagram. I need accurately set up the position in the diagram, which takes some time. And furthermore, it still takes me a pretty long time to translate the moves in my head from the notation to "straight chess moves". As in, for example, if a move is Bf7, it takes me a while to understand that the move is that the bishop moves to f7, and than it takes me even longer to understand which bishop specifically.
Needless to say, this whole process is just very exhausting for me. Somewhere I've read that you should be able to just read the notation like it's part of the book and see what the moves are in your head. I can do this for maybe 2 moves, and than it just falls apart as the position changes more and more from the starting position. Does anyone have any advice on how I could improve my notation reading in chess so it's less exhausting and more accurate? Should I force myself to see the notation in my head or should I keep plugging the moves into an analysis? Thanks in advance and sorry for the long post.