Should I be worried?

A couple days ago I went to the doctor for my annual physical, where they drew blood for lab tests. Today I got a call that they want me to come back in a couple weeks to have my blood drawn again, because my white blood cell count was higher than normal. The guy asked if I'd been feeling sick, and I told him that I have had a sore throat and runny/stopped up nose for the past couple days. I know white blood cells increase to fight off infections; allergies can cause it too, which I also suffer from. But I'm still worried it could be something worse...