Turns out we need build and deploy in separate files to trigger at different times. Now we build on all pushes, but only deploy on tag pushes.