Of course it matters. You want to give your engines enough time to do detailed calculations. I suggest doing one game a day, giving the engines 24 hours of run time.
Engine Tournament in Arena

Of course it matters. You want to give your engines enough time to do detailed calculations. I suggest doing one game a day, giving the engines 24 hours of run time.
24 hours seems way to excessive since it probably takes engines 5 minutes to come up with the same move as it would if they were given 24 hours.

With Arena, I prefer to use repeating time controls for shorter games. CCRL tests engines at two time controls, 40/4 (40 moves in 4 minutes), and 40/40 (40 moves in 40 minutes). The time control repeats every 40 moves. So if you want to get in as many games as possible in a short time, then you could go with the 40/4 time control.

Use whatever time control you want. Have a 5 0 match, or a 3 0 match or whatever. Have several matches, each with different time controls. Have some matches with increments. 5 2 or 3 1, for example. Have "themed" matches... each game begins with the same opening. Have round robin tournaments or swiss tournaments.
It makes no difference. The choice is yours.
And yes, I agree with joesdaking. One game a day is WAY excessive. That wouldn't be my choice for a time control at all.
Hello!
I want to start an engine tournament on Arena to see which engine will win. Should each match be 5 minutes long, or does it matter at all?