‘AT the time I was into tapestry, a hobby I’d failed to mention to Raquel Welch.” This, I tell Hugh Bonneville as he sits munching cress in the rather grand surroundings of the Balmoral Hotel’s Palm Court in Edinburgh, might be my favourite line in his rather fine new memoir Playing Under the Piano.

“There are many more tatters around than you’d realise,” he says after pausing to swallow. “Simon Russell Beale is a very good tatter [tapestry-maker] and Pip Torrens, who was captain of HMS Bedford – I think it was – in Tomorrow Never Dies.” Ah yes, the James Bond film Bonneville also had a very minor role in.

“We did compare notes. But I’m afraid I still have an unfinished tapestry in a bag somewhere at home which my wife keeps telling me I must finish.” It may take a while. These days Bonneville, star of Downton Abbey and the Paddington movies, and someone who has worked with everyone from Bernard Bresslaw and Judi Dench to the aforementioned Raquel Welch and Julia Roberts, is into apple-pressing. “We had two fantastic apple crops this year.

My wife said, ‘You don’t even drink apple juice.’ I said, ‘No, but it’s just fun doing all the crushing.’” It is late afternoon on Tuesday and Bonneville, clean-shaven and wearing a handsome blue tweed suit, is talking quietly and eloquently about his life and work and the people he has met along the way.

He is marvellous company. Normally on a day off he’d be walking his Tibetan terriers along the South Downs. Instead, he’s in Edinburgh, where he’ll talk about his memoir later this evening. But for now he’s humouring me. I’ve come to ask him about his parents, why acting might not be a proper job and which is the better Paddington movie (spoiler: we both agree it’s the second one).

He wrote the memoir in a couple of months. Which is showing off, quite frankly. Because it’s such a lovely thing, full of good humour and self-deprecation. But it also has its share of revelations (his mother was a spy! Well, sort of). And it’s rather moving, particularly when Bonneville writes about what he calls his father’s “dementia journey”. “I was really not anticipating writing about my dad or anything to do with that side of my life. And then it sort of emerged,” he admits.

Throughout, Bonneville argues against the idea that it was inevitable that he would become an actor. And yet he spent his childhood dressing up at every opportunity and watching his older brother and sister on stage. At boarding school in Dorset, while other boys waited to see if they had made the school team, he was desperate to learn if he had passed the audition for the school play.

His future was mapped out early, surely? “I think we all feel, when we look back on our lives, that we can see the dots that join to today. But, really, we’re all living in the present. We can all turn left or right. I could so easily have turned in a different direction.” As a child, he notes in his book, he was both terrified of looking a fool, and thrilled at the idea of stepping into the spotlight.

“And that’s the tension, I suspect, that I have sensed within me ever since,” he writes. That can’t be the case now with more than 30 years in the profession under his belt, can it? “No, it is still there.” Maybe not shyness, Bonneville says.

“I think it’s because my parents, as good natured and encouraging as they were, were very much,‘Don’t show off’. Good middle-class, Church of England values. Everybody else is an expert, but not you.” Actually, his father was an expert. A surgeon. He met Bonneville’s mother when she was a trainee nurse.

“Duty and service, that’s the world they grew up in, in the early days of the NHS. The NHS trained them both and obviously both being in the caring profession it was about giving back to others and not making a fuss.

“Even when my mum was ill later in life and the hospital was frankly pretty flipping negligent, she said, ‘Oh no, don’t make a fuss.’” Bonneville’s childhood was, he admits, a comfortable one. He was in a bubble.

“And so when I joined the National Youth Theatre suddenly I was meeting the coal miner’s son from the north-east, the bank clerk’s daughter from Belfast. It wasn’t a shock. I wasn’t naive. But it was such a pleasure to do plays with people who really wanted to be there.” Did he always feel comfortable on stage? “I think I did. I remember vividly doing this Alan Ayckbourn play and playing this lead part – it was only a 15-minute play – and looking out over my school friends and telling them the story in character and I didn’t feel nervous. I felt confident and I think that just carried on. I felt at home on stage.

“But there’s still the little thing on the shoulder going, ‘You’re not doing a proper job. Go and get a proper job.’” Do actors really think that? Well, yes, it seems. In the book he writes about watching the great Michael Gambon in the National Theatre playing Eddie Carbone in Arthur Miller’s A View From the Bridge. Miller himself said Gambon was “the best Eddie ever”. And yet as the run went on Gambon began to sabotage the performance.

“He would, time and again, turn on the very thing he had created, opting for making his colleagues corpse, as if acting wasn’t a proper job for a man like him, whose passion was engines and mechanics, which he considered to be more real, valuable,” Bonneville records in Playing Under the Piano. “I got to work with him about six years ago and the last time I saw him was about a year ago and he’s frail now and can’t really do it any more,” Bonneville tells me now.

“I know a couple of actors who broke their hearts that he got bored so quickly. When you see a mercurial and individual talent like that … Not that they don’t cherish it, but they doubt it in some way. There is a sort of self-destruct button.

“He didn’t abuse his talent or self-destruct in terms of drink or drugs. He wasn’t an Oliver Reed where you saw this blazing talent who just imploded. “I never talked to him about it when I did know him. Whether he found it just daft …. There was a hint of that about it. Which for me I found really strange when you see that in a talent. Not that it was ever squandered, but maybe slightly downgraded in someone’s own view.

“I’ve seen it once or twice and it surprises me because I am so impressed by talent whether in theatre or music or engineering or whatever.” On the page and in person Bonneville celebrates the talent of the people he has worked with, the likes of Gambon, Judi Dench, Olivia Colman and Kate Winslet.

“There is an immediacy to the way that their thoughts and their emotions connect that is bewitching to watch,” he suggests. “There’s a deep-rooted talent there that sets them apart. It’s like a Federer or a Rafa, they’re just in a different league.” I’m not sure he sees himself on a par and yet Bonneville has been a reliable pleasure on our screens for three decades now.

After years on stage he probably first came to attention with a small part in Notting Hill in 1999. He has hardly been off our screens ever since, though obviously things took quite an upswing when he was cast as Robert Crawley in Downton Abbey in 2010. Does he understand why it became such a phenomenon?

“The answer I customarily give, which has become a bit of a mantra, is simply that when I first read it the characters leapt off the page and I wanted to know what happened next. And then when you add the casting and the locations and the production values, that made a heck of a difference. I think it’s that elusive thing.

“There was a bit of magic dust that landed on it because, structurally, it’s a soap really and the characters are generally benign, the storylines are not going to scare the horses, so I don’t know … “I think ultimately it has to come down to the fact that four generations could watch it together and that really pissed off some critics.” He stops, smiles, picks up some cress. “None of us have got the right answer.” Of course, Downton is not Bonneville’s only success. He also plays Mr Brown in the Paddington movies which, let’s face it, are a cultural highlight of the 21st century.

“I think Paddington could have gone badly wrong in the wrong hands,” Bonneville suggests. “It could have been just a kids’ film. But they are really imaginative pieces of work and, yes, showing a Britain we would love to feel exists, which is a Britain of tolerance and compassion and reaching out the hand of friendship.”

Bonneville is too polite to mention the gap between that fantasy and the current reality. We talk about his parents again. When he was nine his mother announced she was going to start a new office job next to the Lambeth North tube station. She spent a decade working there. Years later Bonneville opened the Evening Standard to find a picture of her office under the headline: “MI6 Building to be sold.”

Did the revelation that his mother worked for MI6 change his perception of her? “Well, it did and it didn’t. She was not a glamorous spy. She was doing filing, albeit of a sensitive nature. Her discretion was paramount. She did take whatever secrets to her grave. My dad said she never talked about it. They were both people of great discretion.” By the time his mother died his father was already disappearing from Bonneville and the rest of the family as his dementia took hold.

When Bonneville’s brother Nigel then passed away suddenly it was difficult to explain the loss to their father. It’s a hard question to ask, but was his father aware that he was, as Bonneville describes it, disappearing into the fog?

“I think he was at peace. He wasn’t a hugger. He found the expression of emotion … I never saw him cry. I didn’t see him cry when mum died. She died in the space of 12 hours but then he was beginning the dementia journey and there is this bizarrely funny moment when my sister and I had to tell him that our brother had died and he was on a mission to talk about the sale of his flat and he didn’t really take it on board.

“And then of course, latterly, when he moved into his care home I kept a beautiful pastel drawing of mum as a young woman and he said, ‘Who is that girl? She is very pretty.’ “We were very fortunate that the way the dementia affected him, it never took away that gentlemanly side of him. He didn’t become aggressive or violent as one hears can happen, so we were very blessed.

But once or twice he got frustrated. He got frustrated most when his hands began to go and he couldn’t play the piano. “The harsh reality of dementia is that it is harder for the carer than the person sometimes unless they come out of the fog so much that they panic and know that they are in this fog. But he just went gradually in one direction.” We never really talk about adults as orphans, do we? It must change your relationship to the world.

“It does actually. People have said you only really become yourself once you’re an orphan and I remember saying to my sister – because obviously we lost our brother as well – it’s only us now.” If you look on IMDb, Hugh Bonneville has 140 credits to his name.

“Is that right? Bloody hell,” he says when I bring this up. Is there anything he hasn’t done that he’d like to? “Gosh, there are a million actors I’d love to work with. I love watching Stephen Graham. I think he’s an amazing actor. I thoroughly enjoy working with Sally Hawkins, Olivia Colman, any of the people I’ve worked with. There are only a couple of people I wouldn’t work with, but they’re quietly off the record.” Any directors? Christopher Nolan, say? “I should be so lucky.” Still, given his success rate, presumably the odd rejection is easier these days. He’s not so sure.

“People might just have two interviews in their life because they go into a career. I do 10 a month sometimes and you’re getting the word ‘no’ most of the time.You have to have thick skin to cope with that. But there’s always a little needle that goes through you.” He tells me a story about the late Ian Holm, star of Chariots of Fire and Alien and one of the most lauded actors of his generation.

“I remember doing a film with Ian 20 years ago. We were in Italy and we were walking back to the hotel and he was looking rather tired and I said, ‘Are you having a break after this?’ And he said, ‘Oh no, I’ve got to go back to New Zealand to do some reshoots on Lord of the Rings and then I’m starting a project in March. And then in June I’m doing this …’ I said, why don’t you just take a break?’ He said, ‘The phone might stop ringing.’

“That sense that you are only as good as your last job and the fear of rejection just stays with you. Oh my God, you’re Sir Ian Holm and you’re … Whatever age he was then, 65? It’s not going to get any easier.” It’s time to wrap up. He has a book event to go to. As we say our goodbyes I mention that a friend had asked me – is Playing Under the Piano a luvvie’s book? Does he hate that word?

“I know it’s pejorative. It’s up there with the word celebrity. Simon Pegg, I think, said being called a celebrity is like having the word c*** written on your forehead. “I know that luvvie gives the impression of the self-indulged, feckless actor and I used to think it’s a fairly daft profession, skipping around in tights. But the other day I had a letter from someone saying, ‘I watched Downton Abbey. It’s the first time I’ve watched it since my mum passed.’ “You get letters like that and people saying, ‘Whatever you may think of the job you do, it’s given us great pleasure and entertainment and also now great memories of my loved one.’

“So, yes, it’s not rocket science and we’re not changing the world. But we are taking people out of the world for an hour or two and actually there is some value in that. “I’m not going to lose sleep over being called a luvvie. I said to my wife very early on – she started calling me luvvie as a nickname – ‘Don’t do that. I get it enough at work or walking down the street.’ “And anyway 24 years later she’s still calling me luvvie.”

Playing Under the Piano by Hugh Bonneville is published by Abacus Books, £22