Bad idea not to quarantine new fish . . .
I used to just do what you do, but then 2/3 of my fish were wiped out a few weeks after adding some new fish. The new fish might have some parasites/etc. in early stages.
Best to quarantine them a few weeks and observe them for strange behavior like laying on the bottom, flashing (quick rubbing against the bottom or an object), etc.