Dynamic port blocking - what the heck?

I'm having a peculiar issue with inbound TCP connections (to my local smart home services and Plex server). Basically, after a period of successful operation, I can no longer receive inbound connections on a given TCP port (it shows as connection refused), and have to change the port mapping to be able to access it again; then, after another period, the new port becomes closed/connection refused, and I have to change it again - although usually the old one works again by this time.

It's definitely upstream of me (I have tested with a laptop on the outside of my router), so any thoughts on what's exactly going on, and how it can be stopped?

-c