Getting the KH up to a more stable level would be a good thing. But what also is going on here is a gas exchange into the water. The water coming out of the tap has less CO2 or O2 or something then the atmosphere. So when the two have balanced out it lowers the pH of the water since there is not KH to buffer. I am sure you could google a article to better explain it, but that is enough to get you started.