Hmm... I thought this one was working properly, but seems I was wrong...

Three hours ago (ish) I activated a garden buff and got Heart Locket lvl 1. I didn't keep track on what happened after that, I was just waiting for the cool down to be done before activate another buff for another garden.
Now I noticed the buffs from population buildings vanished, all but one: https://prnt.sc/n91upj
And of course Heart Locket lvl 1 is not active anymore.

I took another printscreen using stop buff to lighten the active buffs in the area... only one population building still has the buff: https://prnt.sc/n91v6h

30 min later:
I activated another garden buff in another area: https://prnt.sc/n92fea
Heart Locket lvl1 is active again...
Using stop buff to lighten the active buffs in the area: https://prnt.sc/n92esf