From your description, quite likely. I dealt with those in houseplants for a decade. They’re not nearly as destructive as aphids, for instance, but they’re a pain. One time I squeezed a plastic pot that had a plant in it and the larval stage reared its head out of the soil like a sandworm in Dune☠️🪱

Sounds like fungus gnats. They’re a mild hazard to plants and are somewhat annoying.

https://www.thespruce.com/how-i-finally-got-rid-of-fungus-gnats-8678354 Is a pretty decent rundown of what they are and what you can do.