Legal action - false reference provided by last employer?

I'm not very clued up on the legal system (I know defamation of character and libel for damages blah) but not sure if those sit here.

I was looking to leave a my old role, had interviews but ended up having my probation ended because the cottoned on I wanted to leave. I'm under the impression it was this one guy who pushed it. My direct manager had a meeting in my inbox for the next week.

My manager and I got on Grest but this other guy took a dislike towards me out of the blue. In my departure manager wished me luck and said he'd want me back if I wanted to, said he'd give me a great reference and said if have no issue finding another role.

Long. Story. Short.
The other guy provided me with a bad reference when a combat I liked and who really liked me rang up. He took the call because my manager was out and he lied!
He told them I was late every day and if never done a full week in the whole ten months of being there. Absolute BS. I'd been late occasionally yes. But I always worked through linches often manning phones on my own even though that wasn't allowed because they all had lunch together. It was personal and I'm wondering if I have any sort of case on my hands. I didn't get the job but they told me how bad the reference was and they were shocked but had to take his word. He used his managing title to his advantage but he was a division manager, not mine.

I'm from the UK.