The default plugin looks for siblings at the editor, e.g. buttons for new threads or in the editor toolbar. The sibling is always a button, but the PUP creates a DIV container, so it looks a little bit creepy.
If the button is in the toolbar, the PUP creates a DIV container. But all other buttons in the toolbar are dircetly <button></button> container. So the div made some different margins between all other buttons.
I see, don't worry. Next release won't sneak the button in the toolbar.
Also, you should use your own HTML if the default gives you trouble. I made the button to be highly configurable, here: https://v3-docs.chevereto.com/extend/popup-upload-plugin.html and is easy as take the default string and use your own in the script attribute.