There would surely be a limit to how many challenges would be allowed. No sport that uses video replay allows it for every call. In American Sports that have timeouts (Basketball, American Football), the replay often costs a timeout. In Baseball, there is generally only one or two per game allowed. And there are several calls that are not significant enough to allow video review.
For footy I could see maybe a 3 challenge limit. No one would be wasting them on throw-ins or petty calls, but a sending or a dodgy offside call or a pen shout might warrant a challenge.
In those sports there's no limit to how many times it can be used, but you'll find the ref only uses it in cases where it's absolutely needed... you could have 100 calls in a game, but the reality is the ref only uses the technology once or twice.
It's not abused in the sports I've mentioned - it's up to the F.A to put the same style of video ref in place... you'll probably find the ref on the pitch wont over-use the system for the symple fact that he wants all the attention
