When is it okay to not castle?

Couple of things to consider. Castling connects your rooks and develops a rook towards the centre in one move so isnt a loss of tempo even if you aren't concerned about king safety. Consider castling if it looks like the centre might open up, particularly if the e file is half open and definitely if it has completely opened. Also if diagonals towards your king, from the a and h file have opened up (kings diagonals). You can always block a check but then you have a pin to deal with (you did calculate the tactics didnt you? 😁). If pawns in the centre are locked then play is more likely to be on the wings so the king is a bit safer if you decide to leave it centralised. Also king can be safer after the queens have come off but does still depend on the central situation.

Every move you make in chess has value. The pieces have value, your position has value, king safety has value, even the time on the clock has value. With experience you will learn to evaluate all these values againts each other.
At a given time you'll ask yourself what is the value of king safety against attacking chances. As a beginner you should put more value on king safety. As you grow as a player you be able to better evaluate that value and make the right decision.

When you castle, make sure there's enough space, otherwise your king is cornered and can get trapped.
if the queens are traded, it if frequently ok to not bother with it. Depends on the position of course but that is a huge factor.

If you're somewhat new, don't learn openings yet -- learn opening principles. Here's a nice 8 minute video: Chess Basics: Opening Principles
Generally speaking -- *very* generally (and assuming your opponent is completely passive, which is rarely the case) -- you want to get a pawn to d4 and e4, then gets knights out, then bishops, then castle to connect your rooks, then get your rooks towards the center. Tempo is worth spending on castling (presumably your opponent is going to castle at some point, too).

if your king is safest in the middle, no need to castle. If your king is safest on f3, move your king to f3. Do not blindly castle, move your king to where it is safest. Usually, that's when it's castles, but there are many, many exceptions. if ke2 kf3 kg4 is a safer square than castled, then play ke2 kf3 kg4. Put your king where it is safest.