Also despite having a clear advantage in the endgame I managed to blunder so badly, lost my pieces miserably. It's the first time I've played an opponent that's successfully chased down my pieces with their king...
Help with early pawn pushers

Pawn pushing wasn't a problem for you here. You were a lot better for majority of time, all you needed to do is not to let that pawn promote .
First advice - 10 minutes per side is too fast. You will not progress that way. Games will be full of blunders.
You need to know that there will be a lot of mistakes in the beginning even with longer time controls but you will eventually have it under control.
Here is the guide for improvement that you might find useful:
https://www.chess.com/blog/nklristic/the-beginners-tale-first-steps-to-chess-improvement

I agree with nklristic. The reason you lost that game is because you didn't give yourself enough time to think. You are a total beginner and the longest think you had in the entire game was 67.5 seconds on move 9. How do you expect to come up with a plan to take advantage of your opponent's weaknesses in that amount of time?
There are two basic lessons you can take from this game though.
1) You shouldn't try to preserve material you don't need to win.
2) A winning plan in endgames like this is cut the enemy king off using your rook and make a queen.
In this particular game you had a couple of things, where you could have won some material in the opening. For instance after 4. f3 you can play 4. ... Nxe4 5. fxe4 Qh4+ 6. g3 Qxe4 and then win the rook. This is a theme known from the Damiano defence (the same with colors reversed), which is an opening that cannot be recommended.
Another tactic would have been 8. ... Bxb1 9. Rxb1 Nxc3, which forks the rook and the queen in addition to winning a pawn (you executed this a couple of moves later). In general you will have all kinds of tactical opportunities if you're pieces are more developed than you opponent's.
Against the center game (1. e4 e5 2. d4) I would recommend taking the d-pawn. If the queen captures it, you can play Nc6 with tempo. Also in general if you think that pawn chains attacking you make you feel uncomfortable, try to capture some of the pawns with your own pawns. Isolated pawns are much weaker than connected pawns and pose no threat. Another way is to block the pawns. This can be done either by pushing your pawn against your opponent's pawn or placing e.g. a knight in front of a backward pawn (in that game e.g. c3-pawn after b4 and d4 are played), if you can protect your knight on that square. (The knight will be very annoying.)
You had a winning position after the middle game, you just blundered a lot of material and let your opponent promote a pawn.
One bit of advice..
Take time to bring your rooks onto open files. You need more pieces to take advantage of pawn pushers.
I try to just focus on development at first but I feel like I get stressed and start making lots of unforced errors against this type of player in the middlegame.
Here is a game for reference (I am black). I made a series of very stupid blunders in my late middle and endgame but so you have any comments on how I should take advantage of my opponents position when they play like this?
You don't, because that's not the issue. If you are getting stressed and making unforced errors, the problem is not the boardstate, but your reaction to it. You need to give yourself time to think and not overreact. First make sure you do not make moves you know are bad before focusing on making moves that will be excellent. A slower time control would be good for this.
Hello all,
Just getting into chess over the best couple of weeks and am really starting to enjoy it.
I have a lot of trouble with the types of players that push pawns aggressively early on. I try to just focus on development at first but I feel like I get stressed and start making lots of unforced errors against this type of player in the middlegame.
Here is a game for reference (I am black). I made a series of very stupid blunders in my late middle and endgame but so you have any comments on how I should take advantage of my opponents position when they play like this?
Thank you