Can i sue a manager for lying about me?

I recently started a call centre job through an agency and only lasted 2 days. These basically involved sitting through observational training while a member of staff showed me how everything worked, but I never actually got the chance to do any of it. I turned up at work on the Wednesday morning to be told by the Customer Services Manager that I was no longer needed and that I should have got a call from the agency about this. So I rang my recruitment consultant at the agency, who had no idea about me being asked to leave and she said she'd get back to me when she'd talked to the boss onsite. Sure enough she called me back 10 minutes later to say that this manager woman was of the opinion that I wasn't suitable for the job and also that I'd been overheard saying things that were inappropriate for an office. I asked for an explanation on both these points and my consultant said she'd already asked what that meant, and the manager refused to give an explanation. I couldn't think why she would believe I was unsuitable for the role, given that I never got to do anything, just sat and watched someone else working! Also, considering I was too busy concentrating on this to make much smalltalk apart from asking my trainer questions and introducing myself when asked, I didn't see what I could have said that she thought was inappropriate, unless she was just very easily offended, which she hadn't seemed to be. So I decided to call her myself, just in case there was a confidentiality issue or something which meant she could give me the details and not the agency. I rang the company, got put through to the manager herself, and politely explained that I just wanted to know what she meant so that I could avoid making the same mistakes in my next job. All she would say was that I just wasn't suitable, to which I pointed out that this had been established, but how was I unsuitable? She refused point blank to give me an example, so I then asked for an example of inappropriate remarks. Again, she wouldn't tell me anything on the grounds that it was the agency who were employing me, not her. I wouldn't have thought this would make a difference if I really had done something wrong - she was still my onsite supervisor so she should make me aware of any ****-ups! I pointed out that I had asked the agency for her "feedback" and she had refused to tell them what she meant. She said it wasn't her problem. We went round in circles like this for 5 or 10 minutes until I realised this stupid cow was swallowing up my credit. I sarcastically asked her if maybe she didn't want me to improve myself and was she so conservative in her beliefs that she thinks anyone who screws up at work should be haunted by it for the rest of their life and not get another job? She lost her temper with me then and hung up, clearly not used to us 2nd cl citizens, sorry I mean temps, ringing her directly, let alone challenging her bullshit. I don't think I did do anything wrong, I just suspect my face didn't fit and she made stuff up as an excuse to fire me, but surely that's illegal? And if she's accusing me of something, shouldn't she be made to back it up? While I don't actually care if the boss likes me or not, it's not ethical to let it affect her professional decisions. Ironically it's usually hypocrites like that who preach the longest and loudest about work ethics this, leave personal problems at the door that. Can I sue for slander/character defamation? After all, what she's said about me could be potentially quite damaging. I live in the UK if that's any help.