No matter how large the rating difference, a win will earn a positive number (even if websites round it off to zero in some instances, the formula itself awards at least some small fraction of a point every time). After infinite wins the Elo rating would indeed by infinite.
This is ok though, after you understand that Elo is not a measurement like height or speed. It's only useful in certain conditions, like opponents are not hugely mismatched, the players in the pool play often, and a variety of opponents. When this happens the rating is a good predictor for your performance within that population.
Historically there have been times when a person's rating was unreliable. There was a guy named Bloodgood who played only in the prison system, and also faked a bunch of tournament wins to get a really high USCF rating. When it was so high he qualified for the US Championship, they froze his rating.
https://en.wikipedia.org/wiki/Claude_Bloodgood#High_rank_possibly_via_manipulation
Would that not make their hypothetical ELO ratings infinite?